How to utilise timeseries in pandas?
PANDAS CHEATSHEET DATA CLEANING PYTHON DATA MUNGING MACHINE LEARNING RECIPES     ALL TAGS

How to utilise timeseries in pandas?

How to utilise timeseries in pandas?

This recipe helps you utilise timeseries in pandas

0

Recipe Objective

Have you tried to utilise data time or calculate some statistic from date time stamp.

So this is the recipe on how we can utilise timeseries in pandas.

Step 1 - Import the library

from datetime import datetime import pandas as pd

We have imported datetime and pandas which will be needed for the dataset.

Step 2 - Setting up the Data

We have created a dataframe with different features. data = {"date": ["2014-05-01 18:47:05.069722", "2014-05-01 18:47:05.119994", "2014-05-02 18:47:05.178768", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.230071", "2014-05-02 18:47:05.280592", "2014-05-03 18:47:05.332662", "2014-05-03 18:47:05.385109", "2014-05-04 18:47:05.436523", "2014-05-04 18:47:05.486877"], "car_sales": [34, 25, 26, 15, 15, 14, 26, 25, 62, 41]} df = pd.DataFrame(data, columns = ["date", "car_sales"]) print(df)

Step 3 - Dealing with Date Time

Here we will be using different functions that we can use on date time.

  • Converting df["date"] from string to datetime
  • df["date"] = pd.to_datetime(df["date"])
  • Setting df["date"] as the index and delete the column
  • df.index = df["date"] del df["date"] print(); print(df)
  • Viewing all observations that occured in 2014
  • print(df["2014"])
  • Viewing all observations that occured in May 2014
  • print(df["2014-05"])
  • Observations after May 3rd, 2014
  • print(df[datetime(2014, 5, 3):])
  • Observations between May 3rd and May 4th
  • print(df["5/3/2014":"5/4/2014"])
  • Truncation observations after May 2nd 2014
  • print(df.truncate(after="5/3/2014"))
  • Observations of May 2014
  • print(df["5-2014"])
  • Counting the number of observations per timestamp
  • print(df.groupby(level=0).count())
So the output comes as:


                        date  car_sales
0  2014-05-01 18:47:05.069722         34
1  2014-05-01 18:47:05.119994         25
2  2014-05-02 18:47:05.178768         26
3  2014-05-02 18:47:05.230071         15
4  2014-05-02 18:47:05.230071         15
5  2014-05-02 18:47:05.280592         14
6  2014-05-03 18:47:05.332662         26
7  2014-05-03 18:47:05.385109         25
8  2014-05-04 18:47:05.436523         62
9  2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14

                            car_sales
date                                 
2014-05-01 18:47:05.069722         34
2014-05-01 18:47:05.119994         25
2014-05-02 18:47:05.178768         26
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.230071         15
2014-05-02 18:47:05.280592         14
2014-05-03 18:47:05.332662         26
2014-05-03 18:47:05.385109         25
2014-05-04 18:47:05.436523         62
2014-05-04 18:47:05.486877         41

                            car_sales
date                                 
2014-05-01 18:47:05.069722          1
2014-05-01 18:47:05.119994          1
2014-05-02 18:47:05.178768          1
2014-05-02 18:47:05.230071          2
2014-05-02 18:47:05.280592          1
2014-05-03 18:47:05.332662          1
2014-05-03 18:47:05.385109          1
2014-05-04 18:47:05.436523          1
2014-05-04 18:47:05.486877          1

Relevant Projects

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Forecast Inventory demand using historical sales data in R
In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

Data Science Project-TalkingData AdTracking Fraud Detection
Machine Learning Project in R-Detect fraudulent click traffic for mobile app ads using R data science programming language.

Predict Census Income using Deep Learning Models
In this project, we are going to work on Deep Learning using H2O to predict Census income.

Predict Credit Default | Give Me Some Credit Kaggle
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Loan Eligibility Prediction using Gradient Boosting Classifier
This data science in python project predicts if a loan should be given to an applicant or not. We predict if the customer is eligible for loan based on several factors like credit score and past history.

Learn to prepare data for your next machine learning project
Text data requires special preparation before you can start using it for any machine learning project.In this ML project, you will learn about applying Machine Learning models to create classifiers and learn how to make sense of textual data.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Identifying Product Bundles from Sales Data Using R Language
In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.

German Credit Dataset Analysis to Classify Loan Applications
In this data science project, you will work with German credit dataset using classification techniques like Decision Tree, Neural Networks etc to classify loan applications using R.