Solved end-to-end Data Science and Big Data projects

Get ready to use coding projects for solving real-world business problems

Videos

Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

Added this week in “Apache Hadoop Projects”

Data Science Project Design a Hadoop Architecture

Design a Hadoop Architecture

Learn to design Hadoop Architecture and understand how to store data using data acquisition tools in Hadoop.

Data Science Project Real-Time Log Processing using Spark Streaming Architecture

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Data Science Project Create a data pipeline based on messaging using Spark and Hive

Create a data pipeline based on messaging using Spark and Hive

In this spark project, we will simulate a simple real-world batch data pipeline based on messaging using Spark and Hive.

Data Science Project Data Warehouse Design for E-commerce Environments

Data Warehouse Design for E-commerce Environments

In this hive project, you will design a data warehouse for e-commerce environments.

See All Data Science Mini Projects

Added this week in “Apache Hive Projects”

Data Science Project Design a Hadoop Architecture

Design a Hadoop Architecture

Learn to design Hadoop Architecture and understand how to store data using data acquisition tools in Hadoop.

Data Science Project Real-Time Log Processing using Spark Streaming Architecture

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Data Science Project Create a data pipeline based on messaging using Spark and Hive

Create a data pipeline based on messaging using Spark and Hive

In this spark project, we will simulate a simple real-world batch data pipeline based on messaging using Spark and Hive.

Data Science Project Data Warehouse Design for E-commerce Environments

Data Warehouse Design for E-commerce Environments

In this hive project, you will design a data warehouse for e-commerce environments.

See All Data Science Mini Projects

Added this week in “Apache Hbase Projects”

Data Science Project Real-Time Log Processing using Spark Streaming Architecture

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Data Science Project Hadoop Project for Beginners-SQL Analytics with Hive

Hadoop Project for Beginners-SQL Analytics with Hive

In this hadoop project, learn about the features in Hive that allow us to perform analytical queries over large datasets.

Data Science Project Spark Project -Real-time data collection and Spark Streaming Aggregation

Spark Project -Real-time data collection and Spark Streaming Aggregation

In this big data project, we will embark on real-time data collection and aggregation from a simulated real-time system using Spark Streaming.

Data Science Project Implementing OLAP on Hadoop using Apache Kylin

Implementing OLAP on Hadoop using Apache Kylin

In this big data project, we will be performing an OLAP cube design using AdventureWorks database. The deliverable for this session will be to design a cube, build and implement it using Kylin, query the cube and even connect familiar tools (like Excel) with our new cube.

See All Data Science Mini Projects

Added this week in “Apache Pig Projects”

Data Science Project Airline Dataset Analysis using Hadoop, Hive, Pig and Impala

Airline Dataset Analysis using Hadoop, Hive, Pig and Impala

Hadoop Project- Perform basic big data analysis on airline dataset using big data tools -Pig, Hive and Impala.

Data Science Project Process a Million Song Dataset to Predict Song Preferences

Process a Million Song Dataset to Predict Song Preferences

In this big data project, we will discover songs for those artists that are associated with the different cultures across the globe.

See All Data Science Mini Projects

Added this week in “Hadoop HDFS Projects”

Data Science Project Online Hadoop Projects -Solving small file problem in Hadoop

Online Hadoop Projects -Solving small file problem in Hadoop

In this hadoop project, we are going to be continuing the series on data engineering by discussing and implementing various ways to solve the hadoop small file problem.

See All Data Science Mini Projects

Added this week in “Apache Oozie Projects”

Data Science Project Create a data pipeline based on messaging using Spark and Hive

Create a data pipeline based on messaging using Spark and Hive

In this spark project, we will simulate a simple real-world batch data pipeline based on messaging using Spark and Hive.

Data Science Project Web Server Log Processing using Hadoop

Web Server Log Processing using Hadoop

In this hadoop project, you will be using a sample application log file from an application server to a demonstrated scaled-down server log processing pipeline.

Data Science Project Work with Streaming Data using Twitter API to Build a JobPortal

Work with Streaming Data using Twitter API to Build a JobPortal

In this spark streaming project, we are going to build the backend of a IT job ad website by streaming data from twitter for analysis in spark.

See All Data Science Mini Projects

Added this week in “Apache Impala Projects”

Data Science Project Web Server Log Processing using Hadoop

Web Server Log Processing using Hadoop

In this hadoop project, you will be using a sample application log file from an application server to a demonstrated scaled-down server log processing pipeline.

Data Science Project Data Warehouse Design for E-commerce Environments

Data Warehouse Design for E-commerce Environments

In this hive project, you will design a data warehouse for e-commerce environments.

Data Science Project Data processing with Spark SQL

Data processing with Spark SQL

In this Apache Spark SQL project, we will go through provisioning data for retrieval using Spark SQL.

Data Science Project Tough engineering choices with large datasets in Hive Part - 2

Tough engineering choices with large datasets in Hive Part - 2

This is in continuation of the previous Hive project "Tough engineering choices with large datasets in Hive Part - 1", where we will work on processing big data sets using Hive.

See All Data Science Mini Projects

Added this week in “Apache Flume Projects”

Data Science Project Real-Time Log Processing using Spark Streaming Architecture

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Data Science Project Online Hadoop Projects -Solving small file problem in Hadoop

Online Hadoop Projects -Solving small file problem in Hadoop

In this hadoop project, we are going to be continuing the series on data engineering by discussing and implementing various ways to solve the hadoop small file problem.

Data Science Project Real-time Auto Tracking with Spark-Redis

Real-time Auto Tracking with Spark-Redis

Spark Project - Discuss real-time monitoring of taxis in a city. The real-time data streaming will be simulated using Flume. The ingestion will be done using Spark Streaming.

Data Science Project Making real time decision on incoming data using Flume and Kafka

Making real time decision on incoming data using Flume and Kafka

Hadoop Projects for Beginners -Learn data ingestion from a source using Apache Flume and Kafka to make a real-time decision on incoming data.

See All Data Science Mini Projects

Added this week in “Apache Sqoop Projects”

Data Science Project Hadoop Project for Beginners-SQL Analytics with Hive

Hadoop Project for Beginners-SQL Analytics with Hive

In this hadoop project, learn about the features in Hive that allow us to perform analytical queries over large datasets.

Data Science Project Spark Project-Analysis and Visualization on Yelp Dataset

Spark Project-Analysis and Visualization on Yelp Dataset

The goal of this Spark project is to analyze business reviews from Yelp dataset and ingest the final output of data processing in Elastic Search.Also, use the visualisation tool in the ELK stack to visualize various kinds of ad-hoc reports from the data.

See All Data Science Mini Projects

Added this week in “Spark SQL Projects”

Data Science Project Yelp Data Processing Using Spark And Hive Part 1

Yelp Data Processing Using Spark And Hive Part 1

In this big data project, we will continue from a previous hive project "Data engineering on Yelp Datasets using Hadoop tools" and do the entire data processing using spark.

Data Science Project Data processing with Spark SQL

Data processing with Spark SQL

In this Apache Spark SQL project, we will go through provisioning data for retrieval using Spark SQL.

Data Science Project Explore features of Spark SQL in practice on Spark 2.0

Explore features of Spark SQL in practice on Spark 2.0

The goal of this spark project for students is to explore the features of Spark SQL in practice on the latest version of Spark i.e. Spark 2.0.

Data Science Project Data Analysis and Visualisation using Spark and Zeppelin

Data Analysis and Visualisation using Spark and Zeppelin

In this big data project, we will talk about Apache Zeppelin. We will write code, write notes, build charts and share all in one single data analytics environment using Hive, Spark and Pig.

See All Data Science Mini Projects

Added this week in “Spark GraphX Projects”

Data Science Project Design a Network Crawler by Mining Github Social Profiles

Design a Network Crawler by Mining Github Social Profiles

In this big data project, we will look at how to mine and make sense of connections in a simple way by building a Spark GraphX Algorithm and a Network Crawler.

Data Science Project Neo4j Project using Yelp dataset to analyse ratings from users

Neo4j Project using Yelp dataset to analyse ratings from users

In this Neo4j project, you will do network analysis using a graph database to find patterns on how a social network affects business reviews and ratings.

Data Science Project Analysis of Community Interactions using Spark GraphX

Analysis of Community Interactions using Spark GraphX

The goal of this spark project is to analyse the level and strength of interactions across areas of coverage of a telecom provider between different areas in the city of Milan.

See All Data Science Mini Projects

Added this week in “Spark Streaming Projects”

Data Science Project Real-Time Log Processing using Spark Streaming Architecture

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Data Science Project Web Server Log Processing using Hadoop

Web Server Log Processing using Hadoop

In this hadoop project, you will be using a sample application log file from an application server to a demonstrated scaled-down server log processing pipeline.

Data Science Project Analysing Big Data with Twitter Sentiments using Spark Streaming

Analysing Big Data with Twitter Sentiments using Spark Streaming

In this big data spark project, we will do Twitter sentiment analysis using spark streaming on the incoming streaming data.

Data Science Project Hadoop Project-Analysis of Yelp Dataset using Hadoop Hive

Hadoop Project-Analysis of Yelp Dataset using Hadoop Hive

The goal of this hadoop project is to apply some data engineering principles to Yelp Dataset in the areas of processing, storage, and retrieval.

See All Data Science Mini Projects

Added this week in “Spark MLlib Projects”

Data Science Project Analysing Big Data with Twitter Sentiments using Spark Streaming

Analysing Big Data with Twitter Sentiments using Spark Streaming

In this big data spark project, we will do Twitter sentiment analysis using spark streaming on the incoming streaming data.

Data Science Project Spark Project - Airline Dataset Analysis using Spark MLlib

Spark Project - Airline Dataset Analysis using Spark MLlib

In this Hackerday, we will go through the basis of statistics and see how Spark enables us to perform statistical operations like descriptive and inferential statistics over the very large dataset.

See All Data Science Mini Projects

Added this week in “Apache Spark Projects”

Data Science Project PySpark Tutorial - Learn to use Apache Spark with Python

PySpark Tutorial - Learn to use Apache Spark with Python

PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Data Science Project Create a data pipeline based on messaging using Spark and Hive

Create a data pipeline based on messaging using Spark and Hive

In this spark project, we will simulate a simple real-world batch data pipeline based on messaging using Spark and Hive.

Data Science Project Data Warehouse Design for E-commerce Environments

Data Warehouse Design for E-commerce Environments

In this hive project, you will design a data warehouse for e-commerce environments.

Data Science Project Spark Project-Analysis and Visualization on Yelp Dataset

Spark Project-Analysis and Visualization on Yelp Dataset

The goal of this Spark project is to analyze business reviews from Yelp dataset and ingest the final output of data processing in Elastic Search.Also, use the visualisation tool in the ELK stack to visualize various kinds of ad-hoc reports from the data.

See All Data Science Mini Projects

Added this week in “PySpark Projects”

Data Science Project PySpark Tutorial - Learn to use Apache Spark with Python

PySpark Tutorial - Learn to use Apache Spark with Python

PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Data Science Project Insurance Pricing Forecast Using Regression Analysis

Insurance Pricing Forecast Using Regression Analysis

In this project, we are going to talk about insurance forecast by using regression techniques.

See All Data Science Mini Projects

Added this week in “Apache Zepellin Projects”

Data Science Project Data Analysis and Visualisation using Spark and Zeppelin

Data Analysis and Visualisation using Spark and Zeppelin

In this big data project, we will talk about Apache Zeppelin. We will write code, write notes, build charts and share all in one single data analytics environment using Hive, Spark and Pig.

Data Science Project Big Data Hadoop Project-Visualize Daily Wikipedia Trends

Big Data Hadoop Project-Visualize Daily Wikipedia Trends

In this big data project, we'll work with Apache Airflow and write scheduled workflow, which will download data from Wikipedia archives, upload to S3, process them in HIVE and finally analyze on Zeppelin Notebooks.

See All Data Science Mini Projects

Added this week in “Apache Kafka Projects”

Data Science Project Real-Time Log Processing using Spark Streaming Architecture

Real-Time Log Processing using Spark Streaming Architecture

In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Data Science Project Web Server Log Processing using Hadoop

Web Server Log Processing using Hadoop

In this hadoop project, you will be using a sample application log file from an application server to a demonstrated scaled-down server log processing pipeline.

Data Science Project Real-Time Log Processing in Kafka for Streaming Architecture

Real-Time Log Processing in Kafka for Streaming Architecture

The goal of this apache kafka project is to process log entries from applications in real-time using Kafka for the streaming architecture in a microservice sense.

Data Science Project IoT Project-Learn to design an IoT Ready Infrastructure 

IoT Project-Learn to design an IoT Ready Infrastructure 

The goal of this IoT project is to build an argument for generalized streaming architecture for reactive data ingestion based on a microservice architecture. 

See All Data Science Mini Projects

Added this week in “Neo4j Projects”

Data Science Project Neo4j Project using Yelp dataset to analyse ratings from users

Neo4j Project using Yelp dataset to analyse ratings from users

In this Neo4j project, you will do network analysis using a graph database to find patterns on how a social network affects business reviews and ratings.

Data Science Project Modeling & Thinking in Graphs(Neo4J) using Movielens Dataset

Modeling & Thinking in Graphs(Neo4J) using Movielens Dataset

In this big data project using Neo4j, we will be remodelling the movielens dataset in a graph structure and using that structures to answer questions in different ways.

See All Data Science Mini Projects

Added this week in “Redis Projects”

Data Science Project Real-time Auto Tracking with Spark-Redis

Real-time Auto Tracking with Spark-Redis

Spark Project - Discuss real-time monitoring of taxis in a city. The real-time data streaming will be simulated using Flume. The ingestion will be done using Spark Streaming.

See All Data Science Mini Projects

Big Data Projects

Every year, people looking to begin their big data career run into a familiar conundrum - 

"How can I land a big data job with limited experience in this field?".

For an emerging field like big data, finding internships or full-time big data jobs requires you to showcase relevant achievements working with popular open source big data tools like, Hadoop, Spark, Kafka, Pig, Hive, and more. Big data and project-based learning are a perfect fit. The best way to get started is to begin working on diverse big data project titles under the mentorship of industry experts. Professionals will love working on these big data projects because it's like a secret. There is so much practical learning involved you don't realize it. DeZyre's big data projects are perfect for beginners, college students, engineering students, professionals wanting to make a career switch and anyone who wants to master big data skills with hands-on experience. 

 

 

Big Data Projects for Beginners

If you have graduate degree in analytics or relevant field from a top-tier college, it is easy for you to get a big data job. Employers believe that you will be able to add value to their business because of the prestige of the college that has awarded you the degree, and the reality that it is in a subject that is relevant to the kind of skills they are looking for. If you do not have an analytics degree from a top-tier college then you need to build that trust yourself that you have the big data skills that the employer is looking for. The best way to build trust with the hiring manager is to work on interesting big data project ideas and build a portfolio of multiple big data projects - Hadoop projects, spark projects, hive projects, Kafka projects, impala projects, and more. The more "real-world" the big data projects are, the more the hiring manager will trust that you will be an asset to their organization , and the greater are your chances of landing the big data job. The best thing about big data careers is that the work you do on building diverse big data projects often looks exactly similar to the work you will do once you are hired.

For IT professionals or anybody with basic big data knowledge, Dezyre's mini projects on big data will help them take responsibility in solving challenging data problems, and help gain expertise on the popular big data tools like Hadoop, Spark, Hive, Pig,

Big Data Projects for Engineering Students

The good news for people in search of big data projects for CSE students is that there are couple of websites that have big data projects with source code. If you google for search terms like "big data projects GitHub" or "big data projects Quora", you might find suggestions on multiple big data project titles, however, for students on the hunt for big data final year projects, titles and source code is not what all they need for learning. Students need industry expert guidance for deeper understanding and greater retention of knowledge so that they can apply what they know to new real-world big data problems. DeZyre has an excellent project-based learning platform where students will enjoy using a spectrum of big data tools under expert guidance.

Here are some popular big data project titles among the college students-


IT professionals and college students rate our big data projects as exceptional. Whether you are looking to upgrade your skills or you are looking to learn about the complete end-to-end implementation of various big data tools like Hadoop, spark, pig , hive, Kafka, and more, Dezyre's mini projects on big data are just what you want.

What will you get when you enroll for DeZyres Big Data projects?