How to utilise timeseries in pandas?

This recipe helps you utilise timeseries in pandas

Recipe Objective

Have you tried to utilise data time or calculate some statistic from date time stamp.

So this is the recipe on how we can utilise timeseries in pandas.

Get Access to Time Series Analysis Real World Projects in Python

Step 1 - Import the library

from datetime import datetime import pandas as pd

We have imported datetime and pandas which will be needed for the dataset.

Step 2 - Setting up the Data

We have created a dataframe with different features. data = {"date": ["2014-05-01 18:47:05.069722", "2014-05-01 18:47:05.119994", "2014-05-02 18:47:05.178768", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.280592", "2014-05-03 18:47:05.332662", "2014-05-03 18:47:05.385109", "2014-05-04 18:47:05.436523", "2014-05-04 18:47:05.486877"], "car_sales": [34, 25, 26, 15, 15, 14, 26, 25, 62, 41]} df = pd.DataFrame(data, columns = ["date", "car_sales"]) print(df)

Step 3 - Dealing with Date Time

Here we will be using different functions that we can use on date time.

    • Converting df["date"] from string to datetime

df["date"] = pd.to_datetime(df["date"])

    • Setting df["date"] as the index and delete the column

df.index = df["date"] del df["date"] print(); print(df)

    • Viewing all observations that occured in 2014

print(df["2014"])

    • Viewing all observations that occured in May 2014

print(df["2014-05"])

    • Observations after May 3rd, 2014

print(df[datetime(2014, 5, 3):])

    • Observations between May 3rd and May 4th

print(df["5/3/2014":"5/4/2014"])

    • Truncation observations after May 2nd 2014

print(df.truncate(after="5/3/2014"))

    • Observations of May 2014

print(df["5-2014"])

    • Counting the number of observations per timestamp

print(df.groupby(level=0).count())

So the output comes as:

                        date  car_sales
0  2014-05-01 18:47:05.069722         34
1  2014-05-01 18:47:05.119994         25
2  2014-05-02 18:47:05.178768         26
3  2014-05-02 18:47:05.230071         15
4  2014-05-02 18:47:05.230071         15
5  2014-05-02 18:47:05.280592         14
6  2014-05-03 18:47:05.332662         26
7  2014-05-03 18:47:05.385109         25
8  2014-05-04 18:47:05.436523         62
9  2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722          1
2014-05-01 18:47:05.119994          1
2014-05-02 18:47:05.178768          1
2014-05-02 18:47:05.230071          2
2014-05-02 18:47:05.280592          1
2014-05-03 18:47:05.332662          1
2014-05-03 18:47:05.385109          1
2014-05-04 18:47:05.436523          1
2014-05-04 18:47:05.486877          1

Download Materials

What Users are saying..

profile image

Ameeruddin Mohammed

ETL (Abintio) developer at IBM
linkedin profile url

I come from a background in Marketing and Analytics and when I developed an interest in Machine Learning algorithms, I did multiple in-class courses from reputed institutions though I got good... Read More

Relevant Projects

Build a Logistic Regression Model in Python from Scratch
Regression project to implement logistic regression in python from scratch on streaming app data.

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.

AWS Project to Build and Deploy LSTM Model with Sagemaker
In this AWS Sagemaker Project, you will learn to build a LSTM model on Sagemaker for sales forecasting while analyzing the impact of weather conditions on Sales.

Build a Multi-Class Classification Model in Python on Saturn Cloud
In this machine learning classification project, you will build a multi-class classification model in Python on Saturn Cloud to predict the license status of a business.

Build ARCH and GARCH Models in Time Series using Python
In this Project we will build an ARCH and a GARCH model using Python

Build a CNN Model with PyTorch for Image Classification
In this deep learning project, you will learn how to build an Image Classification Model using PyTorch CNN

BigMart Sales Prediction ML Project in Python
The goal of the BigMart Sales Prediction ML project is to build and evaluate different predictive models and determine the sales of each product at a store.

Build CNN for Image Colorization using Deep Transfer Learning
Image Processing Project -Train a model for colorization to make grayscale images colorful using convolutional autoencoders.

Build a Review Classification Model using Gated Recurrent Unit
In this Machine Learning project, you will build a classification model in python to classify the reviews of an app on a scale of 1 to 5 using Gated Recurrent Unit.

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.