Databricks ,the team that created Apache Spark has announced addition of new capabilities that will simplify deployment of spark in the cloud. This new enhancement complements existing data science environment of Databricks that lets users analyze data in real-time with data science notebooks which can then be deployed as Apache Spark jobs in production.Ali Ghodsi, CEO and Co-Founder of Databricks said there is an unforeseen demand for a robust and secure apache spark platform in the cloud to run production workloads.
Apache Spark, an open source framework, known for its speed of processing, and ETL related jobs, which constitutes around 55 percent of the reported use today, has been given a new dimension to its capabilities and, all thanks to IBM z/OS platform. This platform enables Apache Spark to run innately on mainframe systems, providing the users to analyze data on the system itself, which reduces the time it takes to move it to Hadoop for ETL jobs, in turns saving lots of money. Apart from this, Spark can be applied on mainframes in various other ways too, for fraud pattern detection, real time payment status, and targeted marketing to name a few. The main advantage of spark on mainframe is not only its speed, but accessing the data natively and analyzing the data on mainframe itself.(Source: https://www.rtinsights.com/ibm-system-z-apache-spark/)
In a very short span of time Spark has become the most active project in big data with over 1K contributors, from 250 different organizations. With all these credentials on Spark’s name, Databricks, founded by the creators of Apache Spark, has closed $60 million funding from NEA, making it to achieve the $100 million mark. Powered by Apache Spark, Databricks indigenous just-in time data analytics platform on the cloud is now serving to around 400 clients providing data integration, real time data analysis and etc.(Source:https://icrunchdata.com/databricks-raises-60m-fuel-apache-spark/)