Each project comes with 2-5 hours of micro-videos explaining the solution.
Get access to 50+ solved projects with iPython notebooks and datasets.
Add project experience to your Linkedin/Github profiles.
The era of IOT brought with it the need to stream data, process and sometimes display its information in real or near-real time.
In this spark streaming project, we will be using a dataset that passes for real-time data sensor feeds for tracking auto vehicles around the city of Bejing. We will track each vehicle as the signal is received from our streaming simulation (using Flume). We will receive the streams of data using Spark Streaming and use the Redis as a pub/sub middleware.
Furthermore, we will use a java swing based application to display real-time information about all vehicles being tracked. While tracking the vehicle, we will be looking for indexes like current speed, total time and distance covered.
While this spark project is about tracking autos, the principles shared in this big data project will cover wide areas of implementing real-time sensor data processing and much more IOT.
In this hadoop project, you will be using a sample application log file from an application server to a demonstrated scaled-down server log processing pipeline.
In this hadoop project, we are going to be continuing the series on data engineering by discussing and implementing various ways to solve the hadoop small file problem.
Hadoop Projects for Beginners -Learn data ingestion from a source using Apache Flume and Kafka to make a real-time decision on incoming data.