IoT Project-Learn to design an IoT Ready Infrastructure 

IoT Project-Learn to design an IoT Ready Infrastructure 

The goal of this IoT project is to build an argument for generalized streaming architecture for reactive data ingestion based on a microservice architecture. 


Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

What will you learn

A general architecture for smart IoT infrastructure
Understanding the roadmap of the project
Streaming architectures (Lambda & Kappa)
Difference between Arduino and Raspberry pie
MQTT as lightweight messaging protocol and the IOT
Understanding different SmartPipe tech and implementation software
Using Redis as Real-time-auto-tracking , etc. other than Cache system and understanding its limit
Understanding the concept of Sensor chains
Arranging the data of different components of IoT pipeline
Making the decision about MQTT-Spark Streaming vs Kafka-Spark Streaming
Configuring code to start all servers simultaneously
Creating your own virtual environment
Storing Kafka metadata causing Zookeeper
Injecting the data and performing basic EDA
Using Kafka as a data hub for streaming architecture
Extracting Data from each sensor from the HBASE and creating a chainbase
HBase and Spark Integration using the Spark HBase connector
Troubleshooting Redis and server connection
Creating a simulation environment
Using caching server to improve system performance

Project Description

In this IoT project, we will be discussing a general architecture for building smart IOT infrastructure. With the trending advance of IOT in our every facet of life, technology has enabled us to be able to handle a large amount of data ingested with high velocity. This big data project discusses IoT architecture with a sample use case.

Our use-case is a fictitious pipeline network system called SmartPipeNet which is a network of sensors with a back office control system that can monitor pipeline flow and react to events along the various branches to give production feedback, detect and reactively reduce loss, and avoid accidents. It major features include:

  • leakage detection
  • regulate supplies to the various chains of the pipeline based on certain events detection
  • predictive maintenance alerts
  • reactive regulation of pipeline flow including shutdown and restart of flow also on event detection

While we will spend time looking at the implementation as much as the IoT architecture in this big data project, our goal is to build that argument for a generalized streaming architecture for reactive data ingestion based on a microservice architecture. 

Similar Projects

In this big data project, we will continue from a previous hive project "Data engineering on Yelp Datasets using Hadoop tools" and do the entire data processing using spark.

Spark Project - Discuss real-time monitoring of taxis in a city. The real-time data streaming will be simulated using Flume. The ingestion will be done using Spark Streaming.

In this hive project, you will design a data warehouse for e-commerce environments.

Curriculum For This Mini Project

15th July 2017
02h 06m
16th July 2017
01h 45m