How to utilise timeseries in pandas?

This recipe helps you utilise timeseries in pandas

Recipe Objective

Have you tried to utilise data time or calculate some statistic from date time stamp.

So this is the recipe on how we can utilise timeseries in pandas.

Get Access to Time Series Analysis Real World Projects in Python

Step 1 - Import the library

from datetime import datetime import pandas as pd

We have imported datetime and pandas which will be needed for the dataset.

Step 2 - Setting up the Data

We have created a dataframe with different features. data = {"date": ["2014-05-01 18:47:05.069722", "2014-05-01 18:47:05.119994", "2014-05-02 18:47:05.178768", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.280592", "2014-05-03 18:47:05.332662", "2014-05-03 18:47:05.385109", "2014-05-04 18:47:05.436523", "2014-05-04 18:47:05.486877"], "car_sales": [34, 25, 26, 15, 15, 14, 26, 25, 62, 41]} df = pd.DataFrame(data, columns = ["date", "car_sales"]) print(df)

Step 3 - Dealing with Date Time

Here we will be using different functions that we can use on date time.

    • Converting df["date"] from string to datetime

df["date"] = pd.to_datetime(df["date"])

    • Setting df["date"] as the index and delete the column

df.index = df["date"] del df["date"] print(); print(df)

    • Viewing all observations that occured in 2014

print(df["2014"])

    • Viewing all observations that occured in May 2014

print(df["2014-05"])

    • Observations after May 3rd, 2014

print(df[datetime(2014, 5, 3):])

    • Observations between May 3rd and May 4th

print(df["5/3/2014":"5/4/2014"])

    • Truncation observations after May 2nd 2014

print(df.truncate(after="5/3/2014"))

    • Observations of May 2014

print(df["5-2014"])

    • Counting the number of observations per timestamp

print(df.groupby(level=0).count())

So the output comes as:

                        date  car_sales
0  2014-05-01 18:47:05.069722         34
1  2014-05-01 18:47:05.119994         25
2  2014-05-02 18:47:05.178768         26
3  2014-05-02 18:47:05.230071         15
4  2014-05-02 18:47:05.230071         15
5  2014-05-02 18:47:05.280592         14
6  2014-05-03 18:47:05.332662         26
7  2014-05-03 18:47:05.385109         25
8  2014-05-04 18:47:05.436523         62
9  2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722          1
2014-05-01 18:47:05.119994          1
2014-05-02 18:47:05.178768          1
2014-05-02 18:47:05.230071          2
2014-05-02 18:47:05.280592          1
2014-05-03 18:47:05.332662          1
2014-05-03 18:47:05.385109          1
2014-05-04 18:47:05.436523          1
2014-05-04 18:47:05.486877          1

Download Materials

What Users are saying..

profile image

Ray han

Tech Leader | Stanford / Yale University
linkedin profile url

I think that they are fantastic. I attended Yale and Stanford and have worked at Honeywell,Oracle, and Arthur Andersen(Accenture) in the US. I have taken Big Data and Hadoop,NoSQL, Spark, Hadoop... Read More

Relevant Projects

Loan Default Prediction Project using Explainable AI ML Models
Loan Default Prediction Project that employs sophisticated machine learning models, such as XGBoost and Random Forest and delves deep into the realm of Explainable AI, ensuring every prediction is transparent and understandable.

Build CNN Image Classification Models for Real Time Prediction
Image Classification Project to build a CNN model in Python that can classify images into social security cards, driving licenses, and other key identity information.

BERT Text Classification using DistilBERT and ALBERT Models
This Project Explains how to perform Text Classification using ALBERT and DistilBERT

FEAST Feature Store Example for Scaling Machine Learning
FEAST Feature Store Example- Learn to use FEAST Feature Store to manage, store, and discover features for customer churn prediction machine learning project.

Deep Learning Project for Text Detection in Images using Python
CV2 Text Detection Code for Images using Python -Build a CRNN deep learning model to predict the single-line text in a given image.

Multi-Class Text Classification with Deep Learning using BERT
In this deep learning project, you will implement one of the most popular state of the art Transformer models, BERT for Multi-Class Text Classification

LLM Project to Build and Fine Tune a Large Language Model
In this LLM project for beginners, you will learn to build a knowledge-grounded chatbot using LLM's and learn how to fine tune it.

Langchain Project for Customer Support App in Python
In this LLM Project, you will learn how to enhance customer support interactions through Large Language Models (LLMs), enabling intelligent, context-aware responses. This Langchain project aims to seamlessly integrate LLM technology with databases, PDF knowledge bases, and audio processing agents to create a comprehensive customer support application.

Avocado Machine Learning Project Python for Price Prediction
In this ML Project, you will use the Avocado dataset to build a machine learning model to predict the average price of avocado which is continuous in nature based on region and varieties of avocado.

Demand prediction of driver availability using multistep time series analysis
In this supervised learning machine learning project, you will predict the availability of a driver in a specific area by using multi step time series analysis.