Aug 09 2014 10:58 AM
It is cool to see you moving ahead of the course content. Way to go :)
Will talk about Hive next in our course on Monday :).
A small correction, it is not SQL (Hive) it is Hive Query Language (HQL) it is like SQL but not exactly SQL.
Also you will notice that The core of Hadoop is the JAVA APIs that we talked about. As hadoop kept growing in popularity, people found needs to get different auidences to use it and build relevant tools that made it easier for a larger audiences to leverage Hadoop.
So you can notice the evolution that I keep talking about in the class
Core Hadoop API in JAVA --- > Pig Scripts ---> Hive
Manual or script based upload of files -> Flume and Sqoop for Batch upload of semi structured and structured data respectively.
Then to work with Large data sets at the scale that is possible with Hadoop you will See Hbase as a nosql solution in Hadoop.
While all these were fine with Hadoop 1. Hadoop 2 is a complete level higher with a major re-architecture that make the hadoop1 of mechanism batch processes and map reduce irrelevant.
With the hadoop2 re-architecture we will see that hadoop hit a new level of possibilities with Real time data processing, Event based systems, etc.