How to check models f1 score using cross validation in Python?

How to check models f1 score using cross validation in Python?

How to check models f1 score using cross validation in Python?

This recipe helps you check models f1 score using cross validation in Python

Recipe Objective

After training a model we need a measure to check its performance, their are many scoring metric on which we can score the model's performance. Out of many metric we will be using f1 score to measure our models performance. We will also be using cross validation to test the model on multiple sets of data.

This data science python source code does the following:
1. Classification metrics used for validation of model.
2. Performs train_test_split to seperate training and testing dataset
3. Implements CrossValidation on models and calculating the final result using "F1 Score" method.

So this is the recipe on How we can check model's f1-score using cross validation in Python.

Step 1 - Import the library

from sklearn.model_selection import cross_val_score from sklearn.tree import DecisionTreeClassifier from sklearn.datasets import make_classification

We have imported various modules from differnt libraries such as cross_val_score, DecisionTreeClassifier and make_classification.

Step 2 - Setting up the Data

We are generating a dataset with make_classification function which will generate a classification dataset as per the passed parameters. X, y = make_classification(n_samples = 10000, n_features = 3, n_informative = 3, n_redundant = 0, n_classes = 2, random_state = 42)

Step 3 - Model and its accuracy

We are using DecisionTreeClassifier as a model to train the data. We are training the model with cross_validation which will train the data on different training set and it will calculate f1 score for all the test train split.

We are printing the f1 score for all the splits in cross validation and we are also printing mean and standard deviation of f1 score. dec_tree = DecisionTreeClassifier() print(cross_val_score(dec_tree, X, y, scoring="f1", cv = 7)) mean_score = cross_val_score(dec_tree, X, y, scoring="f1", cv = 7).mean() std_score = cross_val_score(dec_tree, X, y, scoring="f1", cv = 7).std() print(mean_score) print(std_score) So the output comes as

[0.92254013 0.91392582 0.93802817 0.92426367 0.93614035 0.92210526
 0.9260539 ]



Download Materials

Relevant Projects

Machine Learning or Predictive Models in IoT - Energy Prediction Use Case
In this machine learning and IoT project, we are going to test out the experimental data using various predictive models and train the models and break the energy usage.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

RASA NLU chatbot creation
The project will use rasa NLU for the Intent classifier, spacy for entity tagging, and mongo dB as the DB. The project will incorporate slot filling and context management and will be supporting the following intent and entities. Intents : product_info | ask_price|cancel_order Entities : product_name|location|order id The project will demonstrate how to generate data on the fly, annotate using framework and how to process those for different pieces of training as discussed above .

Walmart Sales Forecasting Data Science Project
Data Science Project in R-Predict the sales for each department using historical markdown data from the Walmart dataset containing data of 45 Walmart stores.

Digit Recognition using CNN for MNIST Dataset in Python
In this deep learning project, you will build a convolutional neural network using MNIST dataset for handwritten digit recognition.

Ecommerce product reviews - Pairwise ranking and sentiment analysis
This project analyzes a dataset containing ecommerce product reviews. The goal is to use machine learning models to perform sentiment analysis on product reviews and rank them based on relevance. Reviews play a key role in product recommendation systems.

Inventory Demand Forecasting using Machine Learning in R
In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

Credit Card Fraud Detection as a Classification Problem
In this data science project, we will predict the credit card fraud in the transactional dataset using some of the predictive models.

Topic modelling using Kmeans clustering to group customer reviews
In this Kmeans clustering machine learning project, you will perform topic modelling in order to group customer reviews based on recurring patterns.

Learn to prepare data for your next machine learning project
Text data requires special preparation before you can start using it for any machine learning project.In this ML project, you will learn about applying Machine Learning models to create classifiers and learn how to make sense of textual data.