According to Tech Overflow’s latest survey, Spark developers in the US are earning at an average of $125,000 per year. If you take the recent votes on a Stack Overflow survey – Apache Spark has recorded the second highest year-on-year increase at 163.5% on the adoption game.
The new environment called the Data Science Experience hosted on Bluemix cloud platform will have 250 data sets, collaborative workspace and various open source tools for data scientists to speed up analytics in real-time. The Data Science Experience will provide data scientists with a single security-rich managed environment for data curation, data ingestion and data analysis by combining data, content, models and other open source resources like RStudio, Jupyter Notebook, H20 on Apache Spark.
For the complete list of big data companies and their salaries- CLICK HERE
The new Couchbase Spark Connector combines the power of the analytical platform Apache Spark to extract meaningful data insights and operational database platform like Couchbase to turn insights into actions. Couchbase Spark connector will help businesses deliver enriching customer experience through web, mobile and IoT applications. Couchbase Spark connector will add value for use cases like network intrusion detection, failure detection, Customer 360 view, real time product recommendations and fraud detection.
Databricks has made a strategic partnership with the CIA’s investment wing – In-Q-Tel. While In-Q-Tel is a separate entity from the CIA – its history shows how passionate the IQT is when working with world class analytics companies.
It’s not just the well-known vendors like IBM that are betting on Apache Spark but many organizations are adopting Apache Spark for in-memory cluster computing. ClearStory data has unveiled a spark based technology known as IDOD i.e. Infinite Data Overlap Detection. This technology uses data inference and ClearStory data’s harmonization technology (the technology that measures how well the 2 data sets can be combined together). IDOD will help users from a non-technical to mix and match data from various sources and analyse it. IDOD will automate data preparation and blending of the desired data to provide results in minutes when compared to modelling it manually which takes weeks or days.
In the MesosCon 2016 keynotes speech, Matei Zaharia talked about Apache Spark’s advanced data analytics capabilities and its upcoming 2.0 release. The most significant feature in Apache Spark 2.0 is structured streaming. "With structured streaming, you're able to take the data in a stream, build a table in Spark SQL, and serve the table through JBDC, and anything that docks SQL can query the real time state of your stream," Zaharia said.
BlueTalon has released its data centric security solution for Apache Spark. BlueTalon is the first company to provide data security across Hadoop, Spark, hive and other platforms used for big data processing. Data centric security solutions help companies eliminate security blind spots providing them with the ability to control the data layer directly. BlueTalon ensures precise and dynamic security controls with dynamic data masking, data authorization and stealth analytics to protect sensitive data.
Enrol now for Apache Spark Training Online to become a Certified Spark Developer
The new MongoDB connector for Apache Spark will help developers and data scientists glean valuable insights in real-time on operational and streaming data. Industry estimates reveal that 80% of analytics development effort goes into data integration and the new MongoDB connector for spark will eliminate the need to shuttle data between operational and analytics data infrastructure. The new connectors will help developers built applications with ease and faster through a single analytics and database technology stack.
At the Hadoop Summit in San Jose, California, Hortonworks announced several new updates to its big data platform that included enhanced security, easier data analytics using apache spark and better developer productivity. Hortonworks released the availability of Apache Zeppelin, a spark based notebook for data scientists. Apache Zeppelin is a graphical environment that will help data scientists create and share data visualizations. As Zeppelin is built on Apache Spark data scientists can enjoy fast data processing speeds that Spark offers as an in-memory system whilst creating Tableau like data visualizations.