Apache Spark

Star

Apache Spark is an open source distributed general-purpose cluster-computing framework. It provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

Here are 9,632 public repositories matching this topic...

Language: All

Filter by language

All 9,632 Python 2,152 Jupyter Notebook 2,015 Scala 1,950 Java 1,192 Shell 349 HTML 267 JavaScript 186 Dockerfile 125 TypeScript 74 R 58

Sort: Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

apache / spark

Star 42.9k

Apache Spark - A unified analytics engine for large-scale data processing

python java r scala sql big-data spark jdbc

Updated Feb 23, 2026
Scala

DataTalksClub / data-engineering-zoomcamp

Star 38.6k

Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here

docker course kafka spark data-engineering free dbt kestra

Updated Feb 23, 2026
Jupyter Notebook

donnemartin / data-science-ipython-notebooks

Star 28.9k

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

python aws data-science machine-learning caffe theano big-data spark deep-learning hadoop tensorflow numpy scikit-learn keras pandas kaggle scipy matplotlib mapreduce

Updated Mar 20, 2024
Python

getredash / redash

Star 28.2k

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

visualization javascript mysql python bigquery bi spark dashboard athena analytics postgresql business-intelligence redash redshift databricks hacktoberfest spark-sql

Updated Feb 20, 2026
Python

yeasy / docker_practice

Star 25.8k

Zui Xin DockerRong Qi Ji Zhu ,Cong Zhen Shi An Li Zhong Xue Xi Zui Jia Shi Jian Ji Qiao !| Learn and understand Docker&Container technologies, with real DevOps practice!

linux docker kubernetes devops spark book virtualization container mesos swarm cloud-computing

Updated Feb 23, 2026
Go

heibaiying / BigData-Notes

Star 16.9k

Da Shu Ju Ru Men Zhi Nan

phoenix scala kafka big-data spark yarn hive hadoop storm bigdata hbase zookeeper hdfs mapreduce flume azkaban sqoop

Updated Jan 5, 2024
Java

FavioVazquez / ds-cheatsheets

Star 16.2k

List of Data Science Cheatsheets to rule the world

python r programming spark jupyter datascience cheatsheet

Updated Jul 18, 2024

GaiZhenbiao / ChuanhuChatGPT

Star 15.4k

GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.

spark chatbot gemini llama minimax moss gemma claude ernie midjourney chatgpt-api chatglm stablelm ollama qwen dalle3 inspurai

Updated Feb 10, 2026
Python

zhisheng17 / flink-learning

Star 15k

flink learning blog. http://www.54tianzhisheng.cn/ Han Flink Ru Men , Gai Nian , Yuan Li , Shi Zhan , Xing Neng Diao You , Yuan Ma Jie Xi Deng Nei Rong . She Ji Flink Connector, Metrics, Library, DataStream API, Table API & SQL Deng Nei Rong De Xue Xi An Li ,Huan You Flink Luo Di Ying Yong De Da Xing Xiang Mu An Li (PVUV, Ri Zhi Cun Chu , Bai Yi Shu Ju Shi Shi Qu Zhong , Jian Kong Gao Jing )Fen Xiang . Huan Ying Da Jia Zhi Chi Wo De Zhuan Lan <>

mysql redis elasticsearch streaming kafka spark influxdb rabbitmq clickhouse hbase stream-processing opentsdb loki flink rocketmq

Updated Mar 12, 2025
Java

apache / doris

Star 15k

Apache Doris is an easy-to-use, high performance and unified analytics database.

agent bigquery real-time sql database ai spark snowflake olap query-engine redshift dbt elt iceberg hudi delta-lake lakehouse paimon

Updated Feb 22, 2026
Java

aalansehaiyang / technology-talk

Star 14.7k

[(Da Han Mian Shi Zhuan Lan )] Yi Fen JavaCheng Xu Yuan Xu Yao De Ji Zhu Zhi Nan ,Zhe Li You Mian Shi Ti , Xi Tong Jia Gou , Zhi Chang Jin Nang , Zhu Liu Zhong Jian Jian Deng ,Rang Ni Cheng Wei Geng Niu De Zi Ji !

git java kafka spark spring es6 hbase springboot dubbo mycat

Updated Jul 21, 2025

horovod / horovod

Star 14.7k

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

machine-learning spark deep-learning uber mxnet tensorflow mpi keras pytorch machinelearning baidu deeplearning ray

Updated Dec 1, 2025
Python

deeplearning4j / deeplearning4j

Star 14.2k

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...

python java clojure scala spark hadoop gpu intellij linear-algebra artificial-intelligence deeplearning neural-nets dl4j matrix-library deeplearning4j

Updated Feb 20, 2026
Java

wangzhiwubigdata / God-Of-BigData

Star 10.4k

Zhuan Zhu Da Shu Ju Xue Xi Mian Shi ,Da Shu Ju Cheng Shen Zhi Lu Kai Qi . Flink/Spark/Hadoop/Hbase/Hive...

kafka spark hive hadoop bigdata hbase zookeeper hdfs flume flink azkaban

Updated Aug 7, 2023

tobymao / sqlglot

Star 9k

Python SQL Parser and Transpiler

mysql python bigquery parser postgres sql spark presto hive clickhouse sqlite snowflake optimizer transpiler redshift databricks tsql trino sqlparser duckdb

Updated Feb 23, 2026
Python

mage-ai / mage-ai

Star 8.7k

Build, run, and manage data pipelines for integrating and transforming data.

python data-science data machine-learning sql spark pipeline etl pipelines orchestration artificial-intelligence data-engineering data-integration dbt elt transformation data-pipelines reverse-etl

Updated Feb 20, 2026
Python

delta-io / delta

Star 8.6k

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

big-data spark analytics acid delta-lake

Updated Feb 21, 2026
Scala

h2oai / h2o-3

Star 7.5k

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.