The new free Spark Studio IDE allows spark users to build, test and run spark applications on desktop. The IDE is a free cut-down version of the Impetus StreamAnalytix platform and acts as a lightweight development tool for processing data and performing analytics on apache spark data. This IDE can be used to build Spark applications for both batch and streaming mode with ready-to-use operators to drag and drop,select, connect, and configure connectors for a completely functional apache spark pipeline. The IDE can also be used to receive data from local data sources and other data targets and has all the machine learning capabilities of StreamAnalytix platform in a single instance.
(Source -http://www.i-programmer.info/news/197-data-mining/11594-visual-spark-studio-ide-for-spark-apps.html )
If you would like more information about Apache Spark Training and Certification, click the Request Info button on top of this page.
Databricks made the Apache Spark 2.3.0 platform available on the unified analytics platform within a compute engine Databricks Runtime 4.0. Databricks Runtime 4.0 brings in new features that include Machine Learning Model export to simplify production deployments and performance optimizations. Apache Spark 2.3.0 is a major milestone as it introduces the continuous processing mode of structured streaming with millisecond low-latency along with other enhanced features across the project.
(Source - https://globenewswire.com/news-release/2018/03/06/1415871/0/en/Databricks-Announces-Availability-of-Apache-Spark-2-3-Within-its-Unified-Analytics-Platform.html )
Splice Machine, the leading data platform that powers intelligent applications announced that availability of its native Apache Spark DataSource to simplify and accelerate IoT and machine learning applications. This new connector will provide fast, native and ACID compliant datastore for Apache Spark. Data engineers, data scientists and hadoop developers can directly use Apache Spark without excessive data transfers in and out of the Splice Machine.Splice Machine’s underlying Spark engine can be used directly for leveraging advanced spark capabilities such as Spark Streaming, MLlib, and Spark SQL. Moreover, Splice Machine is native to spark so it does not require data serialization and transfer across JDBC/ODBC connections.
(Source : http://www.prweb.com/releases/2018/03/prweb15285756.htm )
Apache Spark has matured as a big data processing framework and become a mainstream solution at ideal time when IoT devices are increasingly growing in the market. IoT devices are expected to drive the Apache Spark market in the next few years with increasing needs for processing large datasets.The demand for Apache Spark framework is anticipated to rise in the near future as fog computing gains popularity.Major players in Apache Spark market are Cloudera, Inc., IBM Corporation, Databricks, MapR Technologies Inc.,and Qubole, Inc.
(Source : https://cmfenews.com/apache-spark-market-iot-devices-anticipated-drive-near-future/ )
Apache Spark 2.3 has been released with two novel features - the biggest and major enhancement is change to the streaming operations since the time Spark Streaming has been added to the Apache Spark project. Spark 2.3 version brings Continuous Processing to Structured Streaming that will provide low-latency responses on the order of 1 millisecond instead of 100 millisecond which you are likely to get with micro-batching.The other major enhancement in 2.3 is the integration of Kubernetes for executing Apache Spark jobs in container clusters.Native integration of Kubernetes will make Hadoop deployments temperamental beasts. Organizations will move their Spark deployments onto Kubernetes either in the cloud or on-premise.
(Source : https://www.infoworld.com/article/3262995/spark/whats-new-in-apache-spark-low-latency-streaming-and-kubernetes.html )
The big data analytics platform based on Apache Spark and Optimized for Azure will help organizations better integrate and scale machine learning projects.AWS continue to lead the cloud space being adopted by 64% firms in 2018, Azure is also gaining popularity with 45% of the organizations using it.Using Microsoft Azure Databricks , the company was able to enhance the productivity of its data science team by more than 50%.Azure Databricks is integrated with other Azure services which include Azure Data Lake Store, Azure SQl Data Warehouse, Azure Cosmos DB, and Azure IoT Hub.
(Source : https://www.techrepublic.com/article/how-microsoft-azure-databricks-can-help-companies-speed-big-data-and-ai-adoption/ )