Airline Dataset Analysis using Hadoop, Hive, Pig and Impala

Before data on any platform will become an asset to any organization, it has to pass through processing stage to ensure quality and availability. Afterward, that data has to be available to users (both human and system users). The availability of quality data in any organization is the guarantee of the value that data science (in general) will be to that organization. 

We are using the airline on-time performance dataset (flights data csv) to demonstrate these principles and techniques in this hadoop project and we will proceed to answer the below questions -

• When is the best time of day/day of week/time of year to fly to minimize delays?
• Do older planes suffer more delays?
• How does the number of people flying between different locations change over time?

We will also transform the data access model into time series and demonstrate how clients can access data in our big data infrastructure using a simple tool like the Excel spreadsheet.