What is differencing in timeseries and why do we do it?

What is differencing in timeseries and why do we do it?

What is differencing in timeseries and why do we do it?

This recipe explains what is differencing in timeseries and why do we do it


Recipe Objective

Differencing is a method of transforming a time series dataset. It can be used to remove the series dependence on time, so-called temporal dependence. This includes structures like trends and seasonality.

So this recipe is a short example on what is differencing in time series and why do we do it. Let's get started.

Step 1 - Import the library

import numpy as np import pandas as pd import matplotlib.pyplot as plt

Let's pause and look at these imports. Numpy and pandas are general ones. Here matplotlib.pyplot will help us in plotting.

Step 2 - Setup the Data

df = pd.read_csv('https://raw.githubusercontent.com/selva86/datasets/master/a10.csv', parse_dates=['date']).set_index('date')

Here, we have used one time series data from github. Also, we have set our index to date.

Now our dataset is ready.

Step 3 - Defining Difference function

def difference(dataset, interval=1): diff = list() for i in range(interval, len(dataset)): value = dataset[i] - dataset[i - interval] diff.append(value) return diff

We are just taking difference between each adjacement elements for removal of trends. Each differnence is taken and put in one list which is sent back when call on.

Step 4 - Visualizing trend

diff = difference(df.value) plt.plot(diff) plt.show()

Here we have simply calling our defined function. Finally, we are trying to understand the trend in dataset.

Step 5 - Lets look at our dataset now

Once we run the above code snippet, we will see:

Scroll down the ipython file to visualize the output.

Clearly, it can be seen that the trend has been removed from our dataset.

Relevant Projects

Choosing the right Time Series Forecasting Methods
There are different time series forecasting methods to forecast stock price, demand etc. In this machine learning project, you will learn to determine which forecasting method to be used when and how to apply with time series forecasting example.

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Mercari Price Suggestion Challenge Data Science Project
Data Science Project in Python- Build a machine learning algorithm that automatically suggests the right product prices.

Data Science Project on Wine Quality Prediction in R
In this R data science project, we will explore wine dataset to assess red wine quality. The objective of this data science project is to explore which chemical properties will influence the quality of red wines.

Machine Learning or Predictive Models in IoT - Energy Prediction Use Case
In this machine learning and IoT project, we are going to test out the experimental data using various predictive models and train the models and break the energy usage.

Predict Employee Computer Access Needs in Python
Data Science Project in Python- Given his or her job role, predict employee access needs using amazon employee database.

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

Deep Learning with Keras in R to Predict Customer Churn
In this deep learning project, we will predict customer churn using Artificial Neural Networks and learn how to model an ANN in R with the keras deep learning package.

Predict Census Income using Deep Learning Models
In this project, we are going to work on Deep Learning using H2O to predict Census income.