#

spark-batch

Here are 8 public repositories matching this topic...

yennanliu / NYC_Taxi_Pipeline

Design/Implement stream/batch architecture on NYC taxi data | #DE

events scala kafka spark s3 spark-streaming stream-pipeline spark-batch nyc-taxi-dataset

Updated Apr 29, 2021
Scala

anish749 / spark2-etl-examples

A project with examples of using few commonly used data manipulation/processing/transformation APIs in Apache Spark 2.0.0

spark spark-sql sparkscala spark2 spark-batch

Updated Aug 5, 2021
Scala

SelimHorri / spark-application

Java Application, uses Apache Spark, handles batch as well as streaming processing

mysql java spark spark-streaming spark-sql spark-batch dataframes-api

Updated Jan 22, 2021
Java

yennanliu / spark-etl-pipeline

Various data stream/batch process demo with Apache Scala Spark 🚀

docker dockerfile scala twitter spark apache-spark sbt pipeline stream-processing sbt-plugin spark-streaming sbt-assembly spark-sql spark-dataframes spark-batch spark-rdd

Updated Feb 28, 2020
Scala

yennanliu / YelpReviews

Build an end to end data application with Yelp review dataset. (data collect -> DB config -> data ETL -> data dashboard (analysis/ML)

docker data-science machine-learning database spark etl kaggle yelp business-intelligence redash data-processing database-modeling spark-batch

Updated Dec 17, 2019
Python

ziedbrah-source / monitor-road-traffic-BigData

The Road Monitoring System is a real-time software application that utilizes big data technologies to monitor and analyze vehicular data on Tunisian roads. It provides insights into vehicle locations, movements, and real-time analytics. The system offers an effective solution for monitoring cars conditions and detecting potential issues promptly.

flask kafka spark mongodb hadoop spark-streaming spark-batch

Updated Jun 11, 2023
Java

VuBacktracking / data-batch-processing

Batch Data Processing Pipeline using MinIO, Spark, PostgreSQL, Great Expectations, DBT and Dbeaver

spark postgresql minio dbeaver dbt batch-processing spark-batch great-expectations

Updated Sep 1, 2024
Python

RickLeite / learning-batch-processing

Learning batch processing with Pyspark Interface for Apache Spark

spark pyspark batch-processing pyspark-notebook spark-batch pyspark-python

Updated Mar 8, 2023
Jupyter Notebook

Improve this page

Add a description, image, and links to the spark-batch topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the spark-batch topic, visit your repo's landing page and select "manage topics."