IoT Project-Learn to design an IoT Ready Infrastructure 

IoT Project-Learn to design an IoT Ready Infrastructure 

The goal of this IoT project is to build an argument for generalized streaming architecture for reactive data ingestion based on a microservice architecture. 

Videos

Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

Customer Love

Read All Reviews

Arvind Sodhi

VP - Data Architect, CDO at Deutsche Bank

I have extensive experience in data management and data processing. Over the past few years I saw the data management technology transition into the Big Data ecosystem and I needed to follow suit. I... Read More

Camille St. Omer

Artificial Intelligence Researcher, Quora 'Most Viewed Writer in 'Data Mining'

I came to the platform with no experience and now I am knowledgeable in Machine Learning with Python. No easy thing I must say, the sessions are challenging and go to the depths. I looked at graduate... Read More

What will you learn

A general architecture for smart IoT infrastructure
Understanding the roadmap of the project
Streaming architectures (Lambda & Kappa)
Difference between Arduino and Raspberry pie
MQTT as lightweight messaging protocol and the IOT
Understanding different SmartPipe tech and implementation software
Using Redis as Real-time-auto-tracking , etc. other than Cache system and understanding its limit
Understanding the concept of Sensor chains
Arranging the data of different components of IoT pipeline
Making the decision about MQTT-Spark Streaming vs Kafka-Spark Streaming
Configuring code to start all servers simultaneously
Creating your own virtual environment
Storing Kafka metadata causing Zookeeper
Injecting the data and performing basic EDA
Using Kafka as a data hub for streaming architecture
Extracting Data from each sensor from the HBASE and creating a chainbase
HBase and Spark Integration using the Spark HBase connector
Troubleshooting Redis and server connection
Creating a simulation environment
Using caching server to improve system performance

Project Description

In this IoT project, we will be discussing a general architecture for building smart IOT infrastructure. With the trending advance of IOT in our every facet of life, technology has enabled us to be able to handle a large amount of data ingested with high velocity. This big data project discusses IoT architecture with a sample use case.

Our use-case is a fictitious pipeline network system called SmartPipeNet which is a network of sensors with a back office control system that can monitor pipeline flow and react to events along the various branches to give production feedback, detect and reactively reduce loss, and avoid accidents. It major features include:

  • leakage detection
  • regulate supplies to the various chains of the pipeline based on certain events detection
  • predictive maintenance alerts
  • reactive regulation of pipeline flow including shutdown and restart of flow also on event detection

While we will spend time looking at the implementation as much as the IoT architecture in this big data project, our goal is to build that argument for a generalized streaming architecture for reactive data ingestion based on a microservice architecture. 

Similar Projects

Building a Data Warehouse using Spark on Hive

In this hive project , we will build a Hive data warehouse from a raw dataset stored in HDFS and present the data in a relational structure so that querying the data will be natural.

View Project Details

Work with Streaming Data using Twitter API to Build a JobPortal

In this spark streaming project, we are going to build the backend of a IT job ad website by streaming data from twitter for analysis in spark.

View Project Details

SQL vs NoSQL-Choosing the right DBMS for your Project

In this project, we will walk through all the various classes of NoSQL database and try to establish where they are the best fit.

View Project Details

Curriculum For This Mini Project

15th July 2017
02h 06m
16th July 2017
01h 45m