Hadoop Project - Choosing the best SQL-on-Hadoop Engine

Hadoop Project - Choosing the best SQL-on-Hadoop Engine

In this project, we will take a look at three different SQL-on-Hadoop engines - Hive, Phoenix, Impala and Presto.

Videos

Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

What will you learn

Apache Phoenix, how it works and how to install it.
Presto, how it works and how to install it.
Impala and how to use it.
Using SQL-on-Hadoop in data processing framework like Spark.
Compare the performance of these various engines.
Lookup other SQL-on-Hadoop out there.

Project Description

The hype around SQL-on-Hadoop had died down and now people want more from these SQL-on-Hadoop engines. More requirements like real-time queries, support from various file formats, support from user-defined functions and support from various client connectivities.

In this Hackerday, we will take a look at three different SQL-on-Hadoop engines - Hive, Phoenix, Impala, and Presto. While our expectations for hive should be relatively expected, we want to to see what it will take to get to adopt other SQL-on-Hadoop engines in our big data infrastructure.

After this Hackerday session, you should be able to make a choice about these engines, make the choice with a real informed decision and be able to extend these to your data processing infrastructure.

Similar Projects

In this project, we will walk through all the various classes of NoSQL database and try to establish where they are the best fit.

Learn to design Hadoop Architecture and understand how to store data using data acquisition tools in Hadoop.

In this spark project, we will continue building the data warehouse from the previous project Yelp Data Processing Using Spark And Hive Part 1 and will do further data processing to develop diverse data products.

Curriculum For This Mini Project