How to utilise timeseries in pandas?
PANDAS CHEATSHEET DATA CLEANING PYTHON DATA MUNGING MACHINE LEARNING RECIPES     ALL TAGS

How to utilise timeseries in pandas?

How to utilise timeseries in pandas?

This recipe helps you utilise timeseries in pandas

0

Recipe Objective

Have you tried to utilise data time or calculate some statistic from date time stamp.

So this is the recipe on how we can utilise timeseries in pandas.

Step 1 - Import the library

from datetime import datetime import pandas as pd

We have imported datetime and pandas which will be needed for the dataset.

Step 2 - Setting up the Data

We have created a dataframe with different features. data = {"date": ["2014-05-01 18:47:05.069722", "2014-05-01 18:47:05.119994", "2014-05-02 18:47:05.178768", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.280592", "2014-05-03 18:47:05.332662", "2014-05-03 18:47:05.385109", "2014-05-04 18:47:05.436523", "2014-05-04 18:47:05.486877"], "car_sales": [34, 25, 26, 15, 15, 14, 26, 25, 62, 41]} df = pd.DataFrame(data, columns = ["date", "car_sales"]) print(df)

Step 3 - Dealing with Date Time

Here we will be using different functions that we can use on date time.

  • Converting df["date"] from string to datetime
  • df["date"] = pd.to_datetime(df["date"])
  • Setting df["date"] as the index and delete the column
  • df.index = df["date"] del df["date"] print(); print(df)
  • Viewing all observations that occured in 2014
  • print(df["2014"])
  • Viewing all observations that occured in May 2014
  • print(df["2014-05"])
  • Observations after May 3rd, 2014
  • print(df[datetime(2014, 5, 3):])
  • Observations between May 3rd and May 4th
  • print(df["5/3/2014":"5/4/2014"])
  • Truncation observations after May 2nd 2014
  • print(df.truncate(after="5/3/2014"))
  • Observations of May 2014
  • print(df["5-2014"])
  • Counting the number of observations per timestamp
  • print(df.groupby(level=0).count())
So the output comes as:


                        date  car_sales
0  2014-05-01 18:47:05.069722         34
1  2014-05-01 18:47:05.119994         25
2  2014-05-02 18:47:05.178768         26
3  2014-05-02 18:47:05.230071         15
4  2014-05-02 18:47:05.230071         15
5  2014-05-02 18:47:05.280592         14
6  2014-05-03 18:47:05.332662         26
7  2014-05-03 18:47:05.385109         25
8  2014-05-04 18:47:05.436523         62
9  2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722          1
2014-05-01 18:47:05.119994          1
2014-05-02 18:47:05.178768          1
2014-05-02 18:47:05.230071          2
2014-05-02 18:47:05.280592          1
2014-05-03 18:47:05.332662          1
2014-05-03 18:47:05.385109          1
2014-05-04 18:47:05.436523          1
2014-05-04 18:47:05.486877          1

Relevant Projects

Predict Credit Default | Give Me Some Credit Kaggle
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Sequence Classification with LSTM RNN in Python with Keras
In this project, we are going to work on Sequence to Sequence Prediction using IMDB Movie Review Dataset​ using Keras in Python.

Mercari Price Suggestion Challenge Data Science Project
Data Science Project in Python- Build a machine learning algorithm that automatically suggests the right product prices.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Forecast Inventory demand using historical sales data in R
In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

Predict Census Income using Deep Learning Models
In this project, we are going to work on Deep Learning using H2O to predict Census income.

Data Science Project - Instacart Market Basket Analysis
Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.

Demand prediction of driver availability using multistep time series analysis
In this supervised learning machine learning project, you will predict the availability of a driver in a specific area by using multi step time series analysis.

Identifying Product Bundles from Sales Data Using R Language
In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.