How to compare different classification models using logloss in ML and how to pick the best one?

This recipe helps you compare different classification models using logloss in ML and how to pick the best one

Recipe Objective

How to compare different classification models using logloss and how to pick the best one

LOG loss is useful when we have to compare models, It compares the model mainly in two ways by their outputs and their probabilistic outcome.

* To calculate LOG loss the classifier assigns the probability to each class.

* LOG loss starts to measures the uncertainity of the model of every sample and it compares with the true labels and in return penalises the false classification.

* LOG loss has the ability to get defined for two or more labels

* LOG loss nearer to 0 means higher accuracy away from zero means lower accuracy. LOG loss has the range between 0 to infinity.

If there are N samples belonging to M classes :

1.) yij , indicates whether sample i belongs to class j or not

2.) pij , indicates the probability of sample i belonging to class j

The negative sign negates log(yij^) output which is always negative. yij^ outputs a probability (0 - 1). log(x) is nagative if 0 < x < 1.

Step 1- Importing Libraries

from sklearn.model_selection import train_test_split, cross_val_score, cross_val_predict import pandas as pd import numpy as np import seaborn as sns from sklearn.linear_model import LogisticRegression

Step 2- Importing and preparing the dataset.

We will import the dataset directly through seaborn library.

iris = sns.load_dataset('iris') X=iris.drop(columns='species') y=iris['species'] Xtrain, Xtest, ytrain, ytest= train_test_split(X,y, test_size=0.3, random_state=20)

Step 3- Fitting the Model.

We will start the fit the Machine Learning Model.

# Logistic Regression clf_logreg = LogisticRegression() # fit model clf_logreg.fit(Xtrain, ytrain)

Step 4- Calculating the LOG LOSS.

we will calculate the LOG LOSS score.

logloss_logreg = cross_val_score(clf_logreg, Xtrain, ytrain, scoring = 'neg_log_loss').mean() print(logloss_logreg)

What Users are saying..

profile image

Ed Godalle

Director Data Analytics at EY / EY Tech
linkedin profile url

I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More

Relevant Projects

AWS MLOps Project to Deploy Multiple Linear Regression Model
Build and Deploy a Multiple Linear Regression Model in Python on AWS

Build Deep Autoencoders Model for Anomaly Detection in Python
In this deep learning project , you will build and deploy a deep autoencoders model using Flask.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Predictive Analytics Project for Working Capital Optimization
In this Predictive Analytics Project, you will build a model to accurately forecast the timing of customer and supplier payments for optimizing working capital.

Image Classification Model using Transfer Learning in PyTorch
In this PyTorch Project, you will build an image classification model in PyTorch using the ResNet pre-trained model.

Build OCR from Scratch Python using YOLO and Tesseract
In this deep learning project, you will learn how to build your custom OCR (optical character recognition) from scratch by using Google Tesseract and YOLO to read the text from any images.

OpenCV Project for Beginners to Learn Computer Vision Basics
In this OpenCV project, you will learn computer vision basics and the fundamentals of OpenCV library using Python.

Learn How to Build a Linear Regression Model in PyTorch
In this Machine Learning Project, you will learn how to build a simple linear regression model in PyTorch to predict the number of days subscribed.

End-to-End Speech Emotion Recognition Project using ANN
Speech Emotion Recognition using RAVDESS Audio Dataset - Build an Artificial Neural Network Model to Classify Audio Data into various Emotions like Sad, Happy, Angry, and Neutral

Build Multi Class Text Classification Models with RNN and LSTM
In this Deep Learning Project, you will use the customer complaints data about consumer financial products to build multi-class text classification models using RNN and LSTM.