News on Apache Spark - March 2017
Harness Hadoop and Spark for user-friendly BI. Infoworld.com, March 2, 2017
Any big data technology must fit into the workflows, skillsets, habits and requirements of various business users across various enterprises.Datameer is a big data analytics application that exactly does that by harnessing the power of open source technologies Hadoop and Spark for user friendly BI. Datameer runs hadoop and spark by hiding their internal complexity and promoting their usage in enterprise IT environments and various business user scenarios. Datameer creates an abstraction layer over hadoop and spark which integrates them into stable platforms and toolchains which can be used in enterprise business environments.
(Source : http://www.infoworld.com/article/3176099/analytics/harness-hadoop-and-spark-for-user-friendly-bi.html )
If you would like more information about Apache Spark Training and Certification, click the Request Info button on top of this page.
Microsoft Announces Preview of Azure HDInsight 3.6 with Apache Spark 2.1. Dabcc.com, March 3, 2017
Microsoft announced its preview of Azure HDInsight 3.6 to get the feedback from its users on Apache Spark 2.1. Users can try out all the new available in the latest version of open source Spark 2,1 together with rich experience of using notebooks on Azure HDInsight 3.6.
(Source: http://www.dabcc.com/microsoft-announces-preview-of-azure-hdinsight-3-6-with-apache-spark-2-1/ )
What’s In the Pipeline for Apache Spark?. Datanami.com, March 6, 2017
Spark creator Matei Zaharia said that Apache Spark will see several novel features and enhancements to the existing features in 2017. Some of the novel features that are likely to be introduced include better integration with Apache Kafka, standard binary data format, and the capability to run spark on laptop. Of all , the most important and novel feature that still remains a long term goal of the Apache Spark community is automated creation of continuous big data applications in Spark.
(Source : https://www.datanami.com/2017/03/06/whats-pipeline-apache-spark/)
Apache Spark among the 10 skills to learn if you want to advance in a career in tech.BusinessInsider.in, March 9,2017.
Change is the only constant thing in the tech world and if you want to advance your career it is must to stay updated with recent technological advancements.Stack Overflow released the ranking of its technologies based on the demand from employers that grew fastest from 2015 to 2016.According to the Stack Overflow Careers Site , there is an increasing demand for big data professionals with Apache Spark and cloud computing skills. Spark is a tool for analyzing big data at high speed and as organizations start investing in the fundamentals of AI, Spark skills are likely to play a vital role in tech in 2017.So, professionals looking to advance their career in the big data space must learn spark in 2017.
(Source : http://www.businessinsider.in/these-are-the-10-skills-to-learn-if-you-want-to-advance-in-a-career-in-tech/articleshow/57559275.cms?format=slideshow )
Impetus Technologies Announces StreamAnalytix 3.0 Featuring Support for Apache Spark-Based Batch Processing. Yahoo.com, March 14, 2017.
A big data thought leader and software solutions company, Impetus Technologies today announced StreamAnalytix 3.0 which includes support for Apache Spark based batch processing and machine learning features.StreamAnalytix 3.0 will help organizations maximize the performance of their analytical models and achieve profitable business outcomes.
(Source : http://finance.yahoo.com/news/impetus-technologies-announces-streamanalytix-3-120000921.html )
ManageEngine Announces Support for Apache Spark at Strata + Hadoop World in San Jose. Yahoo.com, March 14, 2017
ManageEngine demonstrated latest capabilities of its Applications Manager at the Strata + Hadoop world conference in San Jose. The application performance monitoring solution now supports performance monitoring of Apache Spark applications. This will help the operations and development teams in an organization to gain visibility into the performance of the big data engine as well as business critical big data applications that rely on Spark.
(Source : http://finance.yahoo.com/news/manageengine-announces-support-apache-spark-131500917.html )
Microsoft Spark Connector for Azure DocumentDB Supports Data Science.eweek.com, March 15, 2017.
Microsoft unveiled a new spark connector for Azure DocumentDB at the Strata + Hadoop world conference in San Jose this week. The new spark connector will allow Azure customers to perform data science and extract meaningful insights in real-time. Connecting spark to Azure DocumentDB will speed up the ability of customers to solve fast-moving data science problems where data can be easily persisted and retrieved through DocumentDB.
(Source : http://www.eweek.com/cloud/microsoft-spark-connector-for-azure-documentdb-supports-data-science )
Spark ML: getting closer to the edge to improve latency. SiliconAngle.com,March 15, 2017
Machine Learning and IoT are among the biggest tech trends of today and apache spark- structured streaming is making a huge impact on this but few aspects of the spark framework are not meant to be pushed out to the edge. Spark structured streaming probably cannot be used for live surveying of decisions.Though Apache Spark MLib has the capability to take models trained in Apache Spark and push them out to the edge but this does not make much sense for IoT devices.Models that are just few MB’s in size , do not require a large cluster to do predictions but instead the spark cluster can be used to train the models and the prediction can be done using other tools like Karau.
(Source : http://siliconangle.com/blog/2017/03/15/spark-ml-getting-closer-edge-improve-latency-bigdatasv/ )
Can this tiny SQL engine surpass Spark’s data streaming efforts? SIliconAngle.com, March 22, 2017.
Apache Spark claimed to be the best and easiest engine for data streaming with 2.0 release but now a chinese company Transwarp Technologies Co. is challenging spark with a tiny SQL engine that is 5 to 10 times faster than spark 2.0. The company claims that the tiny SQL engine can stream single event at a time which spark cannot do. They have achieved this by modifying the execution model of Apache spark to be event-driven and tweaking SQL to be smaller and faster.The company draws on Apache Spark for few of its streaming capabilities but adds its own individual components to achieve low-latency streaming one event at a time.
(Source : http://siliconangle.com/blog/2017/03/22/can-event-event-streaming-engine-surpass-sparks-best-effort-date-bigdatasv/)