Spark Project-Measuring US Non-Farm Payroll Forex Impact

Spark Project-Measuring US Non-Farm Payroll Forex Impact

In this spark project, we will measure by how much NFP has triggered moves in past markets.

Videos

Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

What will you learn

Discuss the user objective and real-life purpose of this study.
Setting up the virtual environment in Cloudera VM ware
Understanding different types of trading
Gathering data for market analysis
Cleaning and Creating the final database in HDFS
Writing Queries for creating partitioning the dataset
Building Hive tables in Hue editor
Using Hive to make the data accessible
Writing spark application to structure the data
Filtering desired data and changing the datatypes
Joining different types of tables for collective analysis
Grouping different tables using Groupby function
Extracting the schema form the dataset
Loading time frame aggregator from Scala
Creating Spark executable application
Prepare the basis for a Machine Learning system
Incremental loading of data to increase efficiency
Storing the final solution in Parquet datatype

Project Description

For investors in the currency markets, fundamental analysis means analyzing the market using non-technical sources like news and other events. Furthermore, in today's FX Market, one of the most consistency news pieces that resort in a very sharp movement in the forex market is the US Non-farm payroll(NFP).

In this big data project, we want to measure by how much NFP has triggered moves in past markets. We will analyze the news numbers versus the price movement that it resulted in. This information can be very useful to gauge by how much the market will move in the event of any future news. This will enable investors to make splits seconds decision as to where to enter the market and what target points should be expected in the event of NFP news.

While the data prepared from this big data project using apache spark will be a basis for a machine learning predictive system, we will be focusing on the machine learning portion of this class in other upcoming projects. This class will be on how to gather and prepare data for this purpose of inference and prediction.

Similar Projects

In this project, we will look at running various use cases in the analysis of crime data sets using Apache Spark.

In this project, we will look at Cassandra and how it is suited for especially in a hadoop environment, how to integrate it with spark, installation in our lab environment.

In this PySpark project, you will simulate a complex real-world data pipeline based on messaging. This project is deployed using the following tech stack - NiFi, PySpark, Hive, HDFS, Kafka, Airflow, Tableau and AWS QuickSight.

Curriculum For This Mini Project

24-June-2017
02h 48m
25-June-2017
02h 45m