A survey to understand as to which technologies IT leaders would employ to move faster and agility in 2017 noted Apache Spark as one of the important tool for agility in 2017.The other technologies that stood out in the survey include Puppet, Capriza, Okta and MultiChain.Apache Spark provides agility because of its ability to process large amounts of data faster that can help IT leaders make better business decisions and in a more confident manner.Data processing that earlier required weeks or days can now be completed in just few hours or in real-time using Apache Spark.
(Source : http://www.zdnet.com/article/five-emerging-technologies-for-rapid-digital-transformation/ )
If you would like more information about Apache Spark Training and Certification, click the Request Info button on top of this page.
Diablo technologies announced its partnership with Hortonworks ISV/IHV program that has achieved “Product Integration Certification for the Supermicro® Memory1™ server on the Hortonworks Data Platform (HDP™)”.Diablo’s certification with HDP will not allow customers leverage terabytes of high performance application memory on a well-integrated Spark platform.The various advantages of Apache Spark deployments on Diablo powered Memory 1 include - 60% reduced TCO, more than 2x performance improvement per cluster , 5:1 server consolidation and approximately 389% work per server advantage.
(Source : http://finance.yahoo.com/news/diablo-technologies-joins-hortonworks-partner-151900112.html)
MapR released its latest version of MapR Ecosystem Pack (MEP) program 3.0 that provides enhanced security for Apache Spark, new Spark connectors for MapR DS and HBase, a faster version of Hive and updates and integrations with Drill. The latest version of MEP includes Apache Spark 2.1.0 focusing on enhancements in enterprise-ready stability and security, a native Spark connector for MapR-DB JSON that will ease the process of developing real-time and batch pipelines.
(Source : http://www.dbta.com/Editorial/News-Flashes/New-MapR-Ecosystem-Pack-Optimizes-Security-and-Performance-for-Apache-Spark-117466.aspx )
Databricks rolled out the latest version of its cloud based platform on Spark to target data engineering workloads. The new cloud based data science platform will allow data engineers to combine structured streaming , ETL , SQL and machine learning workloads on Spark. The goal of this platform is to combine secure deployment of multiple data pipelines in production. This new platform will address the increasing demand for data engineers which is hybrid job role between data scientists and data analysts.
(Source : https://www.datanami.com/2017/04/12/databricks-eyes-data-engineers-spark-cloud/ )
600 people participated in the Spark Streaming Innovation Contest. The registrants around the world were competing to build a real-world anomaly detection problem using the visual development platform StreamAnalytix which leverages Apache Spark in batch and streaming modes to create real-time ML applications. The participants were evaluated based on the quality of the application built, extent and quality of StreamAnalytix usage and also on how well the solution was documented. A total of $18,000 prize money was awarded to the winners - Grand prize winner (awarded $10,000) – Venu Kanaparthy, Redlands, California, First runner-up (awarded $5,000) – Anindya Saha, Foster City, California and Second runner-up (awarded $3,000) – Kalyan Janaki, Denver, Colorado.
(Source : http://www.prnewswire.com/news-releases/impetus-technologies-reveals-winners-of-spark-streaming-innovation-contest-300440594.html)
Databricks CEO Ali Ghodsi said that Apache Spark has witnessed increased rate of adoption since 2016. He adds on saying that most of the Fortune 500 companies are looking to make a fast move to do analytics in the cloud due to the following reasons -
i) The cost of cloud services have become extremely competitive.
ii) People are more inclined in moving to the cloud to improve their security.
iii) Rate of innovation is faster in the cloud.
(Source : http://searchdatamanagement.techtarget.com/news/450417161/Spark-processing-engine-more-at-home-in-cloud-Databricks-CEO-says )