How to use GradientBoosting Classifier and Regressor in Python?

This recipe helps you use GradientBoosting Classifier and Regressor in Python

Recipe Objective

Have you ever tried to use GradientBoosting models ie. regressor or classifier. In this we will using both for different dataset.

So this recipe is a short example of how we can use GradientBoosting Classifier and Regressor in Python.

Apply Machine Learning to Demand Forecasting Data Science Problems

Step 1 - Import the library

from sklearn import datasets from sklearn import metrics from sklearn.ensemble import GradientBoostingClassifier from sklearn.ensemble import GradientBoostingRegressor from sklearn.model_selection import train_test_split import matplotlib.pyplot as plt import seaborn as sns plt.style.use("ggplot")

Here we have imported various modules like datasets, GradientBoostingRegressor, GradientBoostingClassifier and test_train_split from differnt libraries. We will understand the use of these later while using it in the in the code snipet.
For now just have a look on these imports.

Step 2 - Setup the Data for classifier

Here we have used datasets to load the inbuilt cancer dataset and we have created objects X and y to store the data and the target value respectively. dataset = datasets.load_breast_cancer() X = dataset.data y = dataset.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

Step 3 - Model and its Score

Here, we are using GradientBoostingClassifier as a Machine Learning model to fit the data. model = GradientBoostingClassifier() model.fit(X_train, y_train) print(model) Now we have predicted the output by passing X_test and also stored real target in expected_y. expected_y = y_test predicted_y = model.predict(X_test) Here we have printed classification report and confusion matrix for the classifier. print(metrics.classification_report(expected_y, predicted_y)) print(metrics.confusion_matrix(expected_y, predicted_y))

Step 4 - Setup the Data for regressor

Here we have used datasets to load the inbuilt boston dataset and we have created objects X and y to store the data and the target value respectively. dataset = datasets.load_boston() X = dataset.data y = dataset.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

Step 5 - Model and its Score

Here, we are using GradientBoostingRegressor as a Machine Learning model to fit the data. model = GradientBoostingRegressor() model.fit(X_train, y_train) print(); print(model) Now we have predicted the output by passing X_test and also stored real target in expected_y. expected_y = y_test predicted_y = model.predict(X_test) Here we have printed r2 score and mean squared log error for the Regressor. print(metrics.r2_score(expected_y, predicted_y)) print(metrics.mean_squared_log_error(expected_y, predicted_y)) plt.figure(figsize=(10,10)) sns.regplot(expected_y, predicted_y, fit_reg=True, scatter_kws={"s": 100})

As an output we get:

GradientBoostingClassifier(criterion="friedman_mse", init=None,
              learning_rate=0.1, loss="deviance", max_depth=3,
              max_features=None, max_leaf_nodes=None,
              min_impurity_decrease=0.0, min_impurity_split=None,
              min_samples_leaf=1, min_samples_split=2,
              min_weight_fraction_leaf=0.0, n_estimators=100,
              n_iter_no_change=None, presort="auto", random_state=None,
              subsample=1.0, tol=0.0001, validation_fraction=0.1,
              verbose=0, warm_start=False)

              precision    recall  f1-score   support

           0       0.98      0.91      0.94        45
           1       0.96      0.99      0.97        98

   micro avg       0.97      0.97      0.97       143
   macro avg       0.97      0.95      0.96       143
weighted avg       0.97      0.97      0.96       143


[[41  4]
 [ 1 97]]

GradientBoostingRegressor(alpha=0.9, criterion="friedman_mse", init=None,
             learning_rate=0.1, loss="ls", max_depth=3, max_features=None,
             max_leaf_nodes=None, min_impurity_decrease=0.0,
             min_impurity_split=None, min_samples_leaf=1,
             min_samples_split=2, min_weight_fraction_leaf=0.0,
             n_estimators=100, n_iter_no_change=None, presort="auto",
             random_state=None, subsample=1.0, tol=0.0001,
             validation_fraction=0.1, verbose=0, warm_start=False)

0.8347320926359012

0.02273902061395565

Download Materials

What Users are saying..

profile image

Ed Godalle

Director Data Analytics at EY / EY Tech
linkedin profile url

I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More

Relevant Projects

Time Series Classification Project for Elevator Failure Prediction
In this Time Series Project, you will predict the failure of elevators using IoT sensor data as a time series classification machine learning problem.

PyTorch Project to Build a LSTM Text Classification Model
In this PyTorch Project you will learn how to build an LSTM Text Classification model for Classifying the Reviews of an App .

PyCaret Project to Build and Deploy an ML App using Streamlit
In this PyCaret Project, you will build a customer segmentation model with PyCaret and deploy the machine learning application using Streamlit.

Build Piecewise and Spline Regression Models in Python
In this Regression Project, you will learn how to build a piecewise and spline regression model from scratch in Python to predict the points scored by a sports team.

Deploy Transformer-BART Model on Paperspace Cloud
In this MLOps Project you will learn how to deploy a Tranaformer BART Model for Abstractive Text Summarization on Paperspace Private Cloud

Machine Learning Project to Forecast Rossmann Store Sales
In this machine learning project you will work on creating a robust prediction model of Rossmann's daily sales using store, promotion, and competitor data.

Many-to-One LSTM for Sentiment Analysis and Text Generation
In this LSTM Project , you will build develop a sentiment detection model using many-to-one LSTMs for accurate prediction of sentiment labels in airline text reviews. Additionally, we will also train many-to-one LSTMs on 'Alice's Adventures in Wonderland' to generate contextually relevant text.

Time Series Forecasting Project-Building ARIMA Model in Python
Build a time series ARIMA model in Python to forecast the use of arrival rate density to support staffing decisions at call centres.

Model Deployment on GCP using Streamlit for Resume Parsing
Perform model deployment on GCP for resume parsing model using Streamlit App.

Medical Image Segmentation Deep Learning Project
In this deep learning project, you will learn to implement Unet++ models for medical image segmentation to detect and classify colorectal polyps.