Recap of Hadoop News for September

Recap of Hadoop News for September

News on Hadoop-September 2016

Apache Hadoop News

HPE adapts Vertica analytical database to world with Hadoop,,September 1, 2016.

To compete in a field of diverse data tools, Vertica 8.0 has expanded its analytical database support for Apache Hadoop and Spark integration and also to enhance Apache Kafka management pipeline. Vertical analytic platform could access hadoop data before but with Vertica 8.0 the analytical engine can now directly work with data stored in hadoop without worrying about data movement.

(Source: )

Survey Reveals Troubleshooting as a Top Challenge to Running Hadoop in Production., September 13,2016.

Pepperdata’s survey of 100 production hadoop users revealed that fast and efficient troubleshooting was a top operations challenge but only after lack of hadoop skills.Based on the survey and increasing demand for improved troubleshooting tools, the company launched Pepperdata Insights Service that’s coupled with granular cluster diagnostics to help operations team reduce troubleshooting by 90% and focus on solving challenging performance problems.

(Source: )

Unravel Data Launches Performance Intelligence Platform,Dramatically Improving Big Data Application,September 13, 2016.

Unravel data launched a full-stack performance intelligence platform for optimizing big data operations. The new big data platform will optimize resource usage, accelerate big data operations and provide operations intelligence. The new platform by Unravel Data automates discovery and problem resolution across various big data technologies like Hadoop ,Spark and Apache Kafka, thereby reducing the time taken to resolve any issue in seconds.


Front Ends and Extensions Take Hadoop in New,September 14,2016.

In the history of big data analytics, high level applications have paved way for various useful connectors and front ends which provided extended functionalities over what the original applications could.For instance, the rise of SQL DB applications opened up the door for database front ends, ushered plug-ins, and connectors.Similarly, the rise of excel spreadsheets gave birth to macros, plugins and other excel extensions. Now, in the era of big data, Hadoop has inspired the growth of its ecosystem with powerful front ends and extensions like -Lens, Twill, Kylin, etc.Apache Hadoop ecosystem is growing rapidly.


Have Your Cake And Eat It: Big Data Without Hadoop.,September 19,2016.

Hadoop has gained popularity in the big data space for large scale data mining and building features like recommendations and personalizations that account for the profitability of a company.All this comes at the cost of Hadoop developers, lots of hardware and IT personnel. Using NoSQL alternative to hadoop for use cases that require data hubs, IoT and real time analytics can save time,money and reduce risk.

(Source :

Big Data Analytics –The Best Career Move in the Coming Years!, September 21,2016.

Experts say that 2016 is the best time to make a career in big data and hadoop and here’s why -

1) Salary: The increasing interest in big data and data science is expanding the wages for gifted experts. A professional with a combination of big data and data science skills can earn an average salary of 13.9 lakhs whilst professionals with only big data skills can earn an average salary of 9.8 lakhs.

For the complete list of big data companies and their salaries- CLICK HERE

2) Used in wide range of Industries: Big data usage across varied industries like finance, retail, e-commerce,real estate is making organizations more information and customer centric leading to increased profitability.

3) Big Data is an integral part of Decision Making:In the digital era,organizations have to take quick decisions to stay ahead in the competitive world.

4) Future for Data Analytics: Undeniably, Big Data has surprised the business and will keep on growing in coming years.

5) Empowering Job titles: Loads of opportunities for dedicated and experienced big data professionals.


The Latest Big Changes in Big Data.,September 22, 2016.

The awareness of Big Data and the progress towards it has grown immensely. This has lead to the deeper innovation of the technology. The big player in the big data world- EMC that provides hadoop consultancy as a part of EMC big data portfolio has collaborated in a billion-dollar deal with Dell and Micro Focus, an UK based consulting company.


SAP officially announces acquisition of big data startup,September 27, 2016.

SAP acquired a big data startup Altiscale that deals with cloud based versions of Hadoop and Spark open source software for storing, processing, and analyzing different types of data. It is said that the deal was closed for more than $125 million.  However,SAP did not disclose the numbers but confirmed about the acquisition. SAP and Altiscale will now work together to integrate their technology. .




Online Hadoop Training

Relevant Projects

Hive Project - Visualising Website Clickstream Data with Apache Hadoop
Analyze clickstream data of a website using Hadoop Hive to increase sales by optimizing every aspect of the customer experience on the website from the first mouse click to the last.

Tough engineering choices with large datasets in Hive Part - 1
Explore hive usage efficiently in this hadoop hive project using various file formats such as JSON, CSV, ORC, AVRO and compare their relative performances

Spark Project -Real-time data collection and Spark Streaming Aggregation
In this big data project, we will embark on real-time data collection and aggregation from a simulated real-time system using Spark Streaming.

Real-Time Log Processing in Kafka for Streaming Architecture
The goal of this apache kafka project is to process log entries from applications in real-time using Kafka for the streaming architecture in a microservice sense.

Tough engineering choices with large datasets in Hive Part - 2
This is in continuation of the previous Hive project "Tough engineering choices with large datasets in Hive Part - 1", where we will work on processing big data sets using Hive.

Airline Dataset Analysis using Hadoop, Hive, Pig and Impala
Hadoop Project- Perform basic big data analysis on airline dataset using big data tools -Pig, Hive and Impala.

Explore features of Spark SQL in practice on Spark 2.0
The goal of this spark project for students is to explore the features of Spark SQL in practice on the latest version of Spark i.e. Spark 2.0.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Real-Time Log Processing using Spark Streaming Architecture
In this Spark project, we are going to bring processing to the speed layer of the lambda architecture which opens up capabilities to monitor application real time performance, measure real time comfort with applications and real time alert in case of security

Hadoop Project-Analysis of Yelp Dataset using Hadoop Hive
The goal of this hadoop project is to apply some data engineering principles to Yelp Dataset in the areas of processing, storage, and retrieval.