1-844-696-6465 (US)        +91 77600 44484        help@dezyre.com

Airline Dataset Analysis using Hadoop, Hive, Pig and Impala

Hadoop Project- Perform basic big data analysis on airline dataset using big data tools -Pig, Hive and Impala.
What are the prerequisites for this project?

What will you learn

  • Data preprocessing with Pig
  • Hive vs. MPP database systems (Hive vs. Impala/Drill)
  • Hive/Impala partitioning and clustering
  • Data compression, tuning and query optimization
  • Using database views to represent data.
  • Building time series data model
  • Visuliazing data using Microsoft Excel via ODBC

Project Description

Before data on any platform will become an asset to any organization, it has to pass through processing stage to ensure quality and availability. Afterward, that data has to be available to users (both human and system users). The availability of quality data in any organization is the guarantee of the value that data science (in general) will be to that organization. 

We are using the airline on-time performance dataset (flights data csv) to demonstrate these principles and techniques in this hadoop project and we will proceed to answer the below questions -

  • When is the best time of day/day of week/time of year to fly to minimize delays?
  • Do older planes suffer more delays?
  • How does the number of people flying between different locations change over time?

We will also transform the data access model into time series and demonstrate how clients can access data in our big data infrastructure using a simple tool like the Excel spreadsheet.



Big Data & Enterprise Software Engineer

I am passionate about software development, databases, data analysis and the android platform. My native language is java but no one has stopped me so far from learning and using angular and node.js. Data and data analysis is thrilling and so are my experiences with SQL on Oracle, Microsoft SQL Server, Postgres and MyS see more...

What is Hackerday?

Stay updated in technology trends by working on projects

Live online coding sessions led by industry experts

Build 2-4 projects a month each lasting 6 hours designed to teach you advanced concepts

Code in groups and connect with your community