Spark Project-Measuring US Non-Farm Payroll Forex Impact

Spark Project-Measuring US Non-Farm Payroll Forex Impact

In this spark project, we will measure by how much NFP has triggered moves in past markets.


Each project comes with 2-5 hours of micro-videos explaining the solution.

Code & Dataset

Get access to 50+ solved projects with iPython notebooks and datasets.

Project Experience

Add project experience to your Linkedin/Github profiles.

Customer Love

Read All Reviews

Hiren Ahir

Microsoft Azure SQL Sever Developer, BI Developer

I'm a Graduate student and came into the job market and found a university degree wasn't sufficient to get a good paying job. I aimed at hottest technology in the market Big Data but the word BigData... Read More

Mike Vogt

Information Architect at Bank of America

I have had a very positive experience. The platform is very rich in resources, and the expert was thoroughly knowledgeable on the subject matter - real world hands-on experience. I wish I had this... Read More

What will you learn

Discuss the user objective and real-life purpose of this study.
Setting up the virtual environment in Cloudera VM ware
Understanding different types of trading
Gathering data for market analysis
Cleaning and Creating the final database in HDFS
Writing Queries for creating partitioning the dataset
Building Hive tables in Hue editor
Using Hive to make the data accessible
Writing spark application to structure the data
Filtering desired data and changing the datatypes
Joining different types of tables for collective analysis
Grouping different tables using Groupby function
Extracting the schema form the dataset
Loading time frame aggregator from Scala
Creating Spark executable application
Prepare the basis for a Machine Learning system
Incremental loading of data to increase efficiency
Storing the final solution in Parquet datatype

Project Description

For investors in the currency markets, fundamental analysis means analyzing the market using non-technical sources like news and other events. Furthermore, in today's FX Market, one of the most consistency news pieces that resort in a very sharp movement in the forex market is the US Non-farm payroll(NFP).

In this big data project, we want to measure by how much NFP has triggered moves in past markets. We will analyze the news numbers versus the price movement that it resulted in. This information can be very useful to gauge by how much the market will move in the event of any future news. This will enable investors to make splits seconds decision as to where to enter the market and what target points should be expected in the event of NFP news.

While the data prepared from this big data project using apache spark will be a basis for a machine learning predictive system, we will be focusing on the machine learning portion of this class in other upcoming projects. This class will be on how to gather and prepare data for this purpose of inference and prediction.

Similar Projects

PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

In this project, we will be building and querying an OLAP Cube for Flight Delays on the Hadoop platform.

In this project, we will look at running various use cases in the analysis of crime data sets using Apache Spark.

Curriculum For This Mini Project

02h 48m
02h 45m