How to utilise timeseries in pandas?

This recipe helps you utilise timeseries in pandas

Recipe Objective

Have you tried to utilise data time or calculate some statistic from date time stamp.

So this is the recipe on how we can utilise timeseries in pandas.

Get Access to Time Series Analysis Real World Projects in Python

Step 1 - Import the library

from datetime import datetime import pandas as pd

We have imported datetime and pandas which will be needed for the dataset.

Step 2 - Setting up the Data

We have created a dataframe with different features. data = {"date": ["2014-05-01 18:47:05.069722", "2014-05-01 18:47:05.119994", "2014-05-02 18:47:05.178768", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.280592", "2014-05-03 18:47:05.332662", "2014-05-03 18:47:05.385109", "2014-05-04 18:47:05.436523", "2014-05-04 18:47:05.486877"], "car_sales": [34, 25, 26, 15, 15, 14, 26, 25, 62, 41]} df = pd.DataFrame(data, columns = ["date", "car_sales"]) print(df)

Step 3 - Dealing with Date Time

Here we will be using different functions that we can use on date time.

    • Converting df["date"] from string to datetime

df["date"] = pd.to_datetime(df["date"])

    • Setting df["date"] as the index and delete the column

df.index = df["date"] del df["date"] print(); print(df)

    • Viewing all observations that occured in 2014

print(df["2014"])

    • Viewing all observations that occured in May 2014

print(df["2014-05"])

    • Observations after May 3rd, 2014

print(df[datetime(2014, 5, 3):])

    • Observations between May 3rd and May 4th

print(df["5/3/2014":"5/4/2014"])

    • Truncation observations after May 2nd 2014

print(df.truncate(after="5/3/2014"))

    • Observations of May 2014

print(df["5-2014"])

    • Counting the number of observations per timestamp

print(df.groupby(level=0).count())

So the output comes as:

                        date  car_sales
0  2014-05-01 18:47:05.069722         34
1  2014-05-01 18:47:05.119994         25
2  2014-05-02 18:47:05.178768         26
3  2014-05-02 18:47:05.230071         15
4  2014-05-02 18:47:05.230071         15
5  2014-05-02 18:47:05.280592         14
6  2014-05-03 18:47:05.332662         26
7  2014-05-03 18:47:05.385109         25
8  2014-05-04 18:47:05.436523         62
9  2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722          1
2014-05-01 18:47:05.119994          1
2014-05-02 18:47:05.178768          1
2014-05-02 18:47:05.230071          2
2014-05-02 18:47:05.280592          1
2014-05-03 18:47:05.332662          1
2014-05-03 18:47:05.385109          1
2014-05-04 18:47:05.436523          1
2014-05-04 18:47:05.486877          1

Download Materials

What Users are saying..

profile image

Ameeruddin Mohammed

ETL (Abintio) developer at IBM
linkedin profile url

I come from a background in Marketing and Analytics and when I developed an interest in Machine Learning algorithms, I did multiple in-class courses from reputed institutions though I got good... Read More

Relevant Projects

Predict Churn for a Telecom company using Logistic Regression
Machine Learning Project in R- Predict the customer churn of telecom sector and find out the key drivers that lead to churn. Learn how the logistic regression model using R can be used to identify the customer churn in telecom dataset.

Linear Regression Model Project in Python for Beginners Part 1
Machine Learning Linear Regression Project in Python to build a simple linear regression model and master the fundamentals of regression for beginners.

Avocado Machine Learning Project Python for Price Prediction
In this ML Project, you will use the Avocado dataset to build a machine learning model to predict the average price of avocado which is continuous in nature based on region and varieties of avocado.

Deploy Transformer BART Model for Text summarization on GCP
Learn to Deploy a Machine Learning Model for the Abstractive Text Summarization on Google Cloud Platform (GCP)

Build a Music Recommendation Algorithm using KKBox's Dataset
Music Recommendation Project using Machine Learning - Use the KKBox dataset to predict the chances of a user listening to a song again after their very first noticeable listening event.

Time Series Python Project using Greykite and Neural Prophet
In this time series project, you will forecast Walmart sales over time using the powerful, fast, and flexible time series forecasting library Greykite that helps automate time series problems.

Detectron2 Object Detection and Segmentation Example Python
Object Detection using Detectron2 - Build a Dectectron2 model to detect the zones and inhibitions in antibiogram images.

Build a Text Generator Model using Amazon SageMaker
In this Deep Learning Project, you will train a Text Generator Model on Amazon Reviews Dataset using LSTM Algorithm in PyTorch and deploy it on Amazon SageMaker.

Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.