How to deal with missing values in a Timeseries in Python?
DATA MUNGING DATA CLEANING PYTHON MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to deal with missing values in a Timeseries in Python?

How to deal with missing values in a Timeseries in Python?

This recipe helps you deal with missing values in a Timeseries in Python

0
This python code does the following: 1. Creates necessary Time-Series dataframe for manipulation. 2.Uses different techniques for filling the missing values. 3.Uses Interpolation method to fill missing values. 4.Forward fill and backward fill in time series.
In [1]:
## How to deal with missing values in a Timeseries in Python 
def Kickstarter_Example_45():
    print()
    print(format('How to deal with missing values in a Timeseries in Python',
                 '*^82'))
    import warnings
    warnings.filterwarnings("ignore")

    # Load libraries
    import pandas as pd
    import numpy as np

    # Create date
    time_index = pd.date_range('28/03/2017', periods=5, freq='M')

    # Create data frame, set index
    df = pd.DataFrame(index=time_index);
    print(df)

    # Create feature with a gap of missing values
    df['Sales'] = [1.0,2.0,np.nan,np.nan,5.0];
    print(); print(df)

    # Interpolate missing values
    df1= df.interpolate();
    print(); print(df1)

    # Forward-fill Missing Values
    df2 = df.ffill();
    print(); print(df2)

    # Backfill Missing Values
    df3 = df.bfill();
    print(); print(df3)

    # Interpolate Missing Values But Only Up One Value
    df4 = df.interpolate(limit=1, limit_direction='forward');
    print(); print(df4)

    # Interpolate Missing Values But Only Up Two Values
    df5 = df.interpolate(limit=2, limit_direction='forward');
    print(); print(df5)

Kickstarter_Example_45()
************How to deal with missing values in a Timeseries in Python*************
Empty DataFrame
Columns: []
Index: [2017-03-31 00:00:00, 2017-04-30 00:00:00, 2017-05-31 00:00:00, 2017-06-30 00:00:00, 2017-07-31 00:00:00]

            Sales
2017-03-31    1.0
2017-04-30    2.0
2017-05-31    NaN
2017-06-30    NaN
2017-07-31    5.0

            Sales
2017-03-31    1.0
2017-04-30    2.0
2017-05-31    3.0
2017-06-30    4.0
2017-07-31    5.0

            Sales
2017-03-31    1.0
2017-04-30    2.0
2017-05-31    2.0
2017-06-30    2.0
2017-07-31    5.0

            Sales
2017-03-31    1.0
2017-04-30    2.0
2017-05-31    5.0
2017-06-30    5.0
2017-07-31    5.0

            Sales
2017-03-31    1.0
2017-04-30    2.0
2017-05-31    3.0
2017-06-30    NaN
2017-07-31    5.0

            Sales
2017-03-31    1.0
2017-04-30    2.0
2017-05-31    3.0
2017-06-30    4.0
2017-07-31    5.0

Relevant Projects

Predict Census Income using Deep Learning Models
In this project, we are going to work on Deep Learning using H2O to predict Census income.

Data Science Project-TalkingData AdTracking Fraud Detection
Machine Learning Project in R-Detect fraudulent click traffic for mobile app ads using R data science programming language.

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

Resume parsing with Machine learning - NLP with Python OCR and Spacy
In this machine learning resume parser example we use the popular Spacy NLP python library for OCR and text classification.

Identifying Product Bundles from Sales Data Using R Language
In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Machine Learning or Predictive Models in IoT - Energy Prediction Use Case
In this machine learning and IoT project, we are going to test out the experimental data using various predictive models and train the models and break the energy usage.

Forecast Inventory demand using historical sales data in R
In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

Zillow’s Home Value Prediction (Zestimate)
Data Science Project in R -Build a machine learning algorithm to predict the future sale prices of homes.