How to use GradientBoosting Classifier and Regressor in Python?
MACHINE LEARNING RECIPES DATA CLEANING PYTHON DATA MUNGING PANDAS CHEATSHEET     ALL TAGS

How to use GradientBoosting Classifier and Regressor in Python?

How to use GradientBoosting Classifier and Regressor in Python?

This recipe helps you use GradientBoosting Classifier and Regressor in Python

Recipe Objective

Have you ever tried to use GradientBoosting models ie. regressor or classifier. In this we will using both for different dataset.

So this recipe is a short example of how we can use GradientBoosting Classifier and Regressor in Python.

Step 1 - Import the library

from sklearn import datasets from sklearn import metrics from sklearn.ensemble import GradientBoostingClassifier from sklearn.ensemble import GradientBoostingRegressor from sklearn.model_selection import train_test_split import matplotlib.pyplot as plt import seaborn as sns plt.style.use("ggplot")

Here we have imported various modules like datasets, GradientBoostingRegressor, GradientBoostingClassifier and test_train_split from differnt libraries. We will understand the use of these later while using it in the in the code snipet.
For now just have a look on these imports.

Step 2 - Setup the Data for classifier

Here we have used datasets to load the inbuilt cancer dataset and we have created objects X and y to store the data and the target value respectively. dataset = datasets.load_breast_cancer() X = dataset.data y = dataset.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

Step 3 - Model and its Score

Here, we are using GradientBoostingClassifier as a Machine Learning model to fit the data. model = GradientBoostingClassifier() model.fit(X_train, y_train) print(model) Now we have predicted the output by passing X_test and also stored real target in expected_y. expected_y = y_test predicted_y = model.predict(X_test) Here we have printed classification report and confusion matrix for the classifier. print(metrics.classification_report(expected_y, predicted_y)) print(metrics.confusion_matrix(expected_y, predicted_y))

Step 4 - Setup the Data for regressor

Here we have used datasets to load the inbuilt boston dataset and we have created objects X and y to store the data and the target value respectively. dataset = datasets.load_boston() X = dataset.data y = dataset.target X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

Step 5 - Model and its Score

Here, we are using GradientBoostingRegressor as a Machine Learning model to fit the data. model = GradientBoostingRegressor() model.fit(X_train, y_train) print(); print(model) Now we have predicted the output by passing X_test and also stored real target in expected_y. expected_y = y_test predicted_y = model.predict(X_test) Here we have printed r2 score and mean squared log error for the Regressor. print(metrics.r2_score(expected_y, predicted_y)) print(metrics.mean_squared_log_error(expected_y, predicted_y)) plt.figure(figsize=(10,10)) sns.regplot(expected_y, predicted_y, fit_reg=True, scatter_kws={"s": 100})

As an output we get:

GradientBoostingClassifier(criterion="friedman_mse", init=None,
              learning_rate=0.1, loss="deviance", max_depth=3,
              max_features=None, max_leaf_nodes=None,
              min_impurity_decrease=0.0, min_impurity_split=None,
              min_samples_leaf=1, min_samples_split=2,
              min_weight_fraction_leaf=0.0, n_estimators=100,
              n_iter_no_change=None, presort="auto", random_state=None,
              subsample=1.0, tol=0.0001, validation_fraction=0.1,
              verbose=0, warm_start=False)

              precision    recall  f1-score   support

           0       0.98      0.91      0.94        45
           1       0.96      0.99      0.97        98

   micro avg       0.97      0.97      0.97       143
   macro avg       0.97      0.95      0.96       143
weighted avg       0.97      0.97      0.96       143


[[41  4]
 [ 1 97]]

GradientBoostingRegressor(alpha=0.9, criterion="friedman_mse", init=None,
             learning_rate=0.1, loss="ls", max_depth=3, max_features=None,
             max_leaf_nodes=None, min_impurity_decrease=0.0,
             min_impurity_split=None, min_samples_leaf=1,
             min_samples_split=2, min_weight_fraction_leaf=0.0,
             n_estimators=100, n_iter_no_change=None, presort="auto",
             random_state=None, subsample=1.0, tol=0.0001,
             validation_fraction=0.1, verbose=0, warm_start=False)

0.8347320926359012

0.02273902061395565

Download Materials

Relevant Projects

Avocado Machine Learning Project Python for Price Prediction
In this ML Project, you will use the Avocado dataset to build a machine learning model to predict the average price of avocado which is continuous in nature based on region and varieties of avocado.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Medical Image Segmentation Deep Learning Project
In this deep learning project, you will learn to implement Unet++ models for medical image segmentation to detect and classify colorectal polyps.

Resume parsing with Machine learning - NLP with Python OCR and Spacy
In this machine learning resume parser example we use the popular Spacy NLP python library for OCR and text classification.

Build a Similar Images Finder with Python, Keras, and Tensorflow
Build your own image similarity application using Python to search and find images of products that are similar to any given product. You will implement the K-Nearest Neighbor algorithm to find products with maximum similarity.

Census Income Data Set Project - Predict Adult Census Income
Use the Adult Income dataset to predict whether income exceeds 50K yr based on census data.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

Churn Prediction in Telecom using Machine Learning in R
Estimating churners before they discontinue using a product or service is extremely important. In this ML project, you will develop a churn prediction model in telecom to predict customers who are most likely subject to churn.

Identifying Product Bundles from Sales Data Using R Language
In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.