How to use Classification Metrics in Python?

This recipe helps you use Classification Metrics in Python

Recipe Objective

In a dataset after applying a Classification model how to evaluate it. There are many metrics that we can use. We will be using accuracy , logarithmic loss and Area under ROC.

So this is the recipe on how we we can use Classification Metrics in Python.

Get Access to Plant Species Identification Project using Machine Learning

Step 1 - Import the library

from sklearn import datasets from sklearn import tree, model_selection, metrics from sklearn.model_selection import train_test_split

We have imported datasets, tree, model_selection and test_train_split which will be needed for the dataset.

Step 2 - Setting up the Data

We have imported inbuilt wine dataset and stored data in x and target in y. We have used to split the data by test train split. Then we have used model_selection.KFold. seed = 42 dataset = datasets.load_breast_cancer() X = dataset.data; y = dataset.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25) kfold = model_selection.KFold(n_splits=10, random_state=seed) kfold = model_selection.KFold(n_splits=10, random_state=seed)

Step 3 - Training model and calculating Metrics

Here we will be using DecisionTreeClassifier as a model model = tree.DecisionTreeClassifier() Now we will be calculating different metrics. We will be using cross validation score to calculate the metrices. So we will be printing the mean and standard deviation of all the scores.

    • Calculating Accuracy

scoring = "accuracy" results = model_selection.cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring) print("Accuracy: ", results.mean()); print("Standard Deviation: ", results.std())

    • Calculating Logarithmic Loss

scoring = "neg_log_loss" results = model_selection.cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring) print("Logloss: ", results.mean()); print("Standard Deviation: ", results.std())

    • Calculating Area under ROC curve

scoring = "roc_auc" results = model_selection.cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring) print(); print("AUC: ", results.mean()); print("Standard Deviation: ", results.std())

So the output comes as:

Accuracy:  0.9248615725359912
Standard Deviation:  0.03454639234547574

Logloss:  -2.675538335423929
Standard Deviation:  1.2623224750420183

AUC:  0.9168731849436718
Standard Deviation:  0.027925303925433888
​

Download Materials

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Build a Multi-Class Classification Model in Python on Saturn Cloud
In this machine learning classification project, you will build a multi-class classification model in Python on Saturn Cloud to predict the license status of a business.

Learn to Build a Neural network from Scratch using NumPy
In this deep learning project, you will learn to build a neural network from scratch using NumPy

House Price Prediction Project using Machine Learning in Python
Use the Zillow Zestimate Dataset to build a machine learning model for house price prediction.

Expedia Hotel Recommendations Data Science Project
In this data science project, you will contextualize customer data and predict the likelihood a customer will stay at 100 different hotel groups.

Build a Speech-Text Transcriptor with Nvidia Quartznet Model
In this Deep Learning Project, you will leverage transfer learning from Nvidia QuartzNet pre-trained models to develop a speech-to-text transcriptor.

Skip Gram Model Python Implementation for Word Embeddings
Skip-Gram Model word2vec Example -Learn how to implement the skip gram algorithm in NLP for word embeddings on a set of documents.

Deep Learning Project for Time Series Forecasting in Python
Deep Learning for Time Series Forecasting in Python -A Hands-On Approach to Build Deep Learning Models (MLP, CNN, LSTM, and a Hybrid Model CNN-LSTM) on Time Series Data.

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .

Azure Deep Learning-Deploy RNN CNN models for TimeSeries
In this Azure MLOps Project, you will learn to perform docker-based deployment of RNN and CNN Models for Time Series Forecasting on Azure Cloud.

Medical Image Segmentation Deep Learning Project
In this deep learning project, you will learn to implement Unet++ models for medical image segmentation to detect and classify colorectal polyps.