How to check models f1 score using cross validation in Python?

This recipe helps you check models f1 score using cross validation in Python

Recipe Objective

After training a model we need a measure to check its performance, their are many scoring metric on which we can score the model's performance. Out of many metric we will be using f1 score to measure our models performance. We will also be using cross validation to test the model on multiple sets of data.

This data science python source code does the following:
1. Classification metrics used for validation of model.
2. Performs train_test_split to seperate training and testing dataset
3. Implements CrossValidation on models and calculating the final result using "F1 Score" method.

So this is the recipe on How we can check model's f1-score using cross validation in Python.

Get Closer To Your Dream of Becoming a Data Scientist with 70+ Solved End-to-End ML Projects

Step 1 - Import the library

from sklearn.model_selection import cross_val_score from sklearn.tree import DecisionTreeClassifier from sklearn.datasets import make_classification

We have imported various modules from differnt libraries such as cross_val_score, DecisionTreeClassifier and make_classification.

Step 2 - Setting up the Data

We are generating a dataset with make_classification function which will generate a classification dataset as per the passed parameters. X, y = make_classification(n_samples = 10000, n_features = 3, n_informative = 3, n_redundant = 0, n_classes = 2, random_state = 42)

Explore More Data Science and Machine Learning Projects for Practice. Fast-Track Your Career Transition with ProjectPro

Step 3 - Model and its accuracy

We are using DecisionTreeClassifier as a model to train the data. We are training the model with cross_validation which will train the data on different training set and it will calculate f1 score for all the test train split. We are printing the f1 score for all the splits in cross validation and we are also printing mean and standard deviation of f1 score. dec_tree = DecisionTreeClassifier() print(cross_val_score(dec_tree, X, y, scoring="f1", cv = 7)) mean_score = cross_val_score(dec_tree, X, y, scoring="f1", cv = 7).mean() std_score = cross_val_score(dec_tree, X, y, scoring="f1", cv = 7).std() print(mean_score) print(std_score) So the output comes as

[0.92254013 0.91392582 0.93802817 0.92426367 0.93614035 0.92210526
 0.9260539 ]

0.9257145721528974

0.006172506932493186

Join Millions of Satisfied Developers and Enterprises to Maximize Your Productivity and ROI with ProjectPro - Read ProjectPro Reviews Now!

Download Materials

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Build Classification Algorithms for Digital Transformation[Banking]
Implement a machine learning approach using various classification techniques in Python to examine the digitalisation process of bank customers.

Build CNN Image Classification Models for Real Time Prediction
Image Classification Project to build a CNN model in Python that can classify images into social security cards, driving licenses, and other key identity information.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Build CI/CD Pipeline for Machine Learning Projects using Jenkins
In this project, you will learn how to create a CI/CD pipeline for a search engine application using Jenkins.

Build a Graph Based Recommendation System in Python -Part 1
Python Recommender Systems Project - Learn to build a graph based recommendation system in eCommerce to recommend products.

Skip Gram Model Python Implementation for Word Embeddings
Skip-Gram Model word2vec Example -Learn how to implement the skip gram algorithm in NLP for word embeddings on a set of documents.

Linear Regression Model Project in Python for Beginners Part 2
Machine Learning Linear Regression Project for Beginners in Python to Build a Multiple Linear Regression Model on Soccer Player Dataset.

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

Build a Face Recognition System in Python using FaceNet
In this deep learning project, you will build your own face recognition system in Python using OpenCV and FaceNet by extracting features from an image of a person's face.

BigMart Sales Prediction ML Project in Python
The goal of the BigMart Sales Prediction ML project is to build and evaluate different predictive models and determine the sales of each product at a store.