Big Data

Curated Resources (27)

Tools (5)

• Apache Spark spark.apache.orgfree

  A multi-language unified analytics engine for large-scale data engineering, data science, and machine learning.

• Apache Hadoop hadoop.apache.orgfree

  A framework that allows for the distributed processing of large data sets across clusters of computers.

• Apache Kafka kafka.apache.orgfree

  A distributed event streaming platform used for high-performance data pipelines, streaming analytics, and data integration.

• Apache Cassandra cassandra.apache.orgfree

  An open-source distributed NoSQL database management system designed to handle large amounts of data across commodity servers.

• Ray ray.iofree

  An open-source unified compute framework for scaling AI and Python workloads across distributed clusters.

Databases (4)

• Common Crawl commoncrawl.orgfree

  An open repository of web crawl data providing petabytes of freely accessible web archives for large-scale analysis.

• Registry of Open Data on AWS registry.opendata.awsfree

  A public registry of large-scale open datasets hosted on Amazon Web Services spanning science, health, and geospatial data.

• Data.gov data.govfree

  The United States government's open data repository providing access to hundreds of thousands of public datasets.

• Kaggle Datasets kaggle.comfree

  A community platform hosting hundreds of thousands of open tabular, text, and multimodal datasets for analytical research.

Learn (6)

• Designing Data-Intensive Applications dataintensive.netpaid

  An authoritative guide by Martin Kleppmann exploring principles of distributed databases, stream processing, and storage systems.

• DataCamp datacamp.comfreemium

  An interactive online learning platform providing courses and certifications in big data, SQL, Python, and data engineering.

• Distributed Systems distributed-systems.netfree

  A free online textbook by Maarten van Steen and Andrew S. Tanenbaum covering principles and paradigms of distributed computing.

• Google Cloud Big Data Documentation cloud.google.comfree

  Official documentation and architectural patterns for petabyte-scale analytics, BigQuery, and distributed streaming engines.

• BigQuery Documentation docs.cloud.google.comfree

  Official documentation for Google BigQuery, the serverless data warehouse, including guides to SQL queries, user-defined functions, loading data and administration.

• Data Engineer Roadmap (roadmap.sh) roadmap.shfree

  Interactive roadmap on data engineering: topics laid out in a suggested study order, each linking to further reading.

Publications (4)

• IEEE Transactions on Big Data computer.orgpaid

  A scholarly quarterly journal publishing research on novel models, architectures, and algorithms for big data.

• ACM Transactions on Database Systems dl.acm.orgfree

  An archival peer-reviewed journal publishing research on the design, implementation, and performance of database systems.

• Proceedings of the VLDB Endowment vldb.orgfree

  A monthly scholarly publication presenting research on database management, big data infrastructure, and data systems.

• Communications of the ACM cacm.acm.orgfree

  The flagship open-access publication of the ACM covering critical developments in computing, distributed systems, and data.

Blogs and sites (4)

• BigDATAwire bigdatawire.comfree

  A news portal offering industry reporting, market analysis, and trends in big data, AI, and advanced analytics.

• KDnuggets kdnuggets.comfree

  A professional resource featuring tutorials, opinion articles, and industry insights in big data, analytics, and data science.

• The Morning Paper blog.acolyer.orgfree

  An archival review blog by Adrian Colyer providing concise, practitioner-oriented summaries of systems and big data research papers.

• Towards Data Science towardsdatascience.comfreemium

  An independent publication sharing technical essays, tutorials, and practical articles on large-scale data engineering.

Events and conferences (4)

• IEEE BigData bigdata.ieee.orgpaid

  An annual flagship IEEE conference presenting research breakthroughs across big data engineering, systems, and algorithms.

• ACM SIGMOD sigmod.orgpaid

  An international annual conference on the management of data and large-scale database systems sponsored by ACM SIGMOD.

• ACM SIGKDD kdd.orgpaid

  A premier international annual conference focusing on knowledge discovery, data mining, and large-scale data science.

• VLDB Conference vldb.orgpaid

  An annual international forum on very large data bases, scalable database management, and distributed data systems.

Other subcategories

Artificial Intelligence · Machine Learning · Computer Information Systems · Computer Science · Computer Vision · Cyber Security · Data Science · Databases · Distributed Computing · Hardware · Human-Computer Interaction · IT Certification · Natural Language Processing · Networking · Operating Systems · Robotics · Software Engineering

Part of Computing