How to evaluate timeseries models using BIC?
MACHINE LEARNING RECIPES DATA CLEANING PYTHON DATA MUNGING PANDAS CHEATSHEET     ALL TAGS

# How to evaluate timeseries models using BIC?

This recipe helps you evaluate timeseries models using BIC

0

## Recipe Objective

The Bayesian Information Criterion (BIC) is an index used in Bayesian statistics to choose between two or more alternative models. Comparing models with the Bayesian information criterion simply involves calculating the BIC for each model. The model with the lowest BIC is considered the best.

So this recipe is a short example on how to evaluate time series models using BIC. Let's get started.

## Step 1 - Import the library

``` import numpy as np import pandas as pd from statsmodels.tsa.arima_model import ARIMA ```

Let's pause and look at these imports. Numpy and pandas are general ones. Here matplotlib.pyplot will help us in plotting. statsmodels.tsa.arima_model will help us in model building.

## Step 2 - Setup the Data

``` df = pd.read_csv('https://raw.githubusercontent.com/selva86/datasets/master/a10.csv', parse_dates=['date']) ```

Here, we have used one time series data from github. Also, we have set our index to date.

## Step 3 - Calculating BIC

``` for i in range(0,2): for j in range(0,2): for k in range(0,2): model = ARIMA(df.value, order=(i, j, k)).fit() print(model.bic) ```

Best BIC can easily be calcuated through libraries. Here we have tried to understand what actually is happening inside. With variation of values of orders, BIC can be seen varying.

## Step 4 - Lets look at our dataset now

Once we run the above code snippet, we will see:

```1316.66388768731
1162.0554484438037
913.5172157072849
868.8258162104078
918.9268418564968
888.116114839384
889.526006058412
857.0907664191164
```

Clearly, order (1,1,1) is best fitted solution to our model. It can be extended further to 2 degrees to have a better understanding of results.

#### Relevant Projects

##### Mercari Price Suggestion Challenge Data Science Project
Data Science Project in Python- Build a machine learning algorithm that automatically suggests the right product prices.

##### Data Science Project on Wine Quality Prediction in R
In this R data science project, we will explore wine dataset to assess red wine quality. The objective of this data science project is to explore which chemical properties will influence the quality of red wines.

##### Human Activity Recognition Using Multiclass Classification in Python
In this human activity recognition project, we use multiclass classification machine learning techniques to analyse fitness dataset from a smartphone tracker.

##### Deep Learning with Keras in R to Predict Customer Churn
In this deep learning project, we will predict customer churn using Artificial Neural Networks and learn how to model an ANN in R with the keras deep learning package.

##### Predict Employee Computer Access Needs in Python
Data Science Project in Python- Given his or her job role, predict employee access needs using amazon employee database.

##### Topic modelling using Kmeans clustering to group customer reviews
In this Kmeans clustering machine learning project, you will perform topic modelling in order to group customer reviews based on recurring patterns.

##### German Credit Dataset Analysis to Classify Loan Applications
In this data science project, you will work with German credit dataset using classification techniques like Decision Tree, Neural Networks etc to classify loan applications using R.

##### Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

##### Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

##### Predict Credit Default | Give Me Some Credit Kaggle
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.