How to do recursive feature elimination in Python (DecisionTreeRegressor)?
FEATURE EXTRACTION DATA CLEANING PYTHON DATA MUNGING MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to do recursive feature elimination in Python (DecisionTreeRegressor)?

How to do recursive feature elimination in Python (DecisionTreeRegressor)?

This recipe helps you do recursive feature elimination in Python (DecisionTreeRegressor)

0

Recipe Objective

While training a model there are many feature which are recursive and it effects the computational cost.

So this is the recipe on how we can do recursive feature elimination in Python (DecisionTreeRegressor).

Step 1 - Importing Library

from sklearn.datasets import make_regression from sklearn.feature_selection import RFECV from sklearn.tree import DecisionTreeRegressor

We have only imported make_regression, RFECV, DecisionTreeRegressor which is needed.

Step 2 - Creating dataset

We have created a dataset by function make_regression with desired parameters. X, y = make_regression(n_samples = 10000, n_features = 100, n_informative = 2) print(X.shape)

Step 3 - Using recursive feature eliminator

.

  • We are using DecisionTreeRegressor as a model
  • dtree = DecisionTreeRegressor()
  • Using Recursive feature elemination. We have passed the required parameters as model, step, scoring, cv and verbose. We have fited the data and transformed it to remove the recursive feature.
  • rfecv = RFECV(estimator=dtree, step=1, scoring="neg_mean_squared_error", cv=4, verbose=1, n_jobs = 4) rfecv.fit(X, y) rfecv.transform(X) print(rfecv) print(rfecv.n_features_) So the output comes as:

    (10000, 100)
    Fitting estimator with 100 features.
    Fitting estimator with 99 features.
    Fitting estimator with 98 features.
    Fitting estimator with 97 features.
    Fitting estimator with 96 features.
    Fitting estimator with 95 features.
    Fitting estimator with 94 features.
    Fitting estimator with 93 features.
    Fitting estimator with 92 features.
    Fitting estimator with 91 features.
    Fitting estimator with 90 features.
    Fitting estimator with 89 features.
    Fitting estimator with 88 features.
    Fitting estimator with 87 features.
    Fitting estimator with 86 features.
    Fitting estimator with 85 features.
    Fitting estimator with 84 features.
    Fitting estimator with 83 features.
    Fitting estimator with 82 features.
    Fitting estimator with 81 features.
    Fitting estimator with 80 features.
    Fitting estimator with 79 features.
    Fitting estimator with 78 features.
    Fitting estimator with 77 features.
    Fitting estimator with 76 features.
    Fitting estimator with 75 features.
    Fitting estimator with 74 features.
    Fitting estimator with 73 features.
    Fitting estimator with 72 features.
    Fitting estimator with 71 features.
    Fitting estimator with 70 features.
    Fitting estimator with 69 features.
    Fitting estimator with 68 features.
    Fitting estimator with 67 features.
    Fitting estimator with 66 features.
    Fitting estimator with 65 features.
    Fitting estimator with 64 features.
    Fitting estimator with 63 features.
    Fitting estimator with 62 features.
    Fitting estimator with 61 features.
    Fitting estimator with 60 features.
    Fitting estimator with 59 features.
    Fitting estimator with 58 features.
    Fitting estimator with 57 features.
    Fitting estimator with 56 features.
    Fitting estimator with 55 features.
    Fitting estimator with 54 features.
    Fitting estimator with 53 features.
    Fitting estimator with 52 features.
    Fitting estimator with 51 features.
    Fitting estimator with 50 features.
    Fitting estimator with 49 features.
    Fitting estimator with 48 features.
    Fitting estimator with 47 features.
    Fitting estimator with 46 features.
    Fitting estimator with 45 features.
    Fitting estimator with 44 features.
    Fitting estimator with 43 features.
    Fitting estimator with 42 features.
    Fitting estimator with 41 features.
    Fitting estimator with 40 features.
    Fitting estimator with 39 features.
    Fitting estimator with 38 features.
    Fitting estimator with 37 features.
    Fitting estimator with 36 features.
    Fitting estimator with 35 features.
    Fitting estimator with 34 features.
    Fitting estimator with 33 features.
    Fitting estimator with 32 features.
    Fitting estimator with 31 features.
    Fitting estimator with 30 features.
    Fitting estimator with 29 features.
    Fitting estimator with 28 features.
    Fitting estimator with 27 features.
    Fitting estimator with 26 features.
    Fitting estimator with 25 features.
    Fitting estimator with 24 features.
    Fitting estimator with 23 features.
    Fitting estimator with 22 features.
    Fitting estimator with 21 features.
    Fitting estimator with 20 features.
    Fitting estimator with 19 features.
    Fitting estimator with 18 features.
    Fitting estimator with 17 features.
    Fitting estimator with 16 features.
    Fitting estimator with 15 features.
    Fitting estimator with 14 features.
    Fitting estimator with 13 features.
    Fitting estimator with 12 features.
    Fitting estimator with 11 features.
    Fitting estimator with 10 features.
    Fitting estimator with 9 features.
    Fitting estimator with 8 features.
    Fitting estimator with 7 features.
    Fitting estimator with 6 features.
    Fitting estimator with 5 features.
    Fitting estimator with 4 features.
    Fitting estimator with 3 features.
    
    RFECV(cv=4,
       estimator=DecisionTreeRegressor(criterion="mse", max_depth=None, max_features=None,
               max_leaf_nodes=None, min_impurity_decrease=0.0,
               min_impurity_split=None, min_samples_leaf=1,
               min_samples_split=2, min_weight_fraction_leaf=0.0,
               presort=False, random_state=None, splitter="best"),
       min_features_to_select=1, n_jobs=4, scoring="neg_mean_squared_error",
       step=1, verbose=1)
    
    2
    
    

Relevant Projects

Predict Macro Economic Trends using Kaggle Financial Dataset
In this machine learning project, you will uncover the predictive value in an uncertain world by using various artificial intelligence, machine learning, advanced regression and feature transformation techniques.

Sequence Classification with LSTM RNN in Python with Keras
In this project, we are going to work on Sequence to Sequence Prediction using IMDB Movie Review Dataset​ using Keras in Python.

Learn to prepare data for your next machine learning project
Text data requires special preparation before you can start using it for any machine learning project.In this ML project, you will learn about applying Machine Learning models to create classifiers and learn how to make sense of textual data.

Time Series Forecasting with LSTM Neural Network Python
Deep Learning Project- Learn to apply deep learning paradigm to forecast univariate time series data.

Music Recommendation System Project using Python and R
Machine Learning Project - Work with KKBOX's Music Recommendation System dataset to build the best music recommendation engine.

Zillow’s Home Value Prediction (Zestimate)
Data Science Project in R -Build a machine learning algorithm to predict the future sale prices of homes.

Human Activity Recognition Using Smartphones Data Set
In this deep learning project, you will build a classification system where to precisely identify human fitness activities.

German Credit Dataset Analysis to Classify Loan Applications
In this data science project, you will work with German credit dataset using classification techniques like Decision Tree, Neural Networks etc to classify loan applications using R.

Ecommerce product reviews - Pairwise ranking and sentiment analysis
This project analyzes a dataset containing ecommerce product reviews. The goal is to use machine learning models to perform sentiment analysis on product reviews and rank them based on relevance. Reviews play a key role in product recommendation systems.

Loan Eligibility Prediction using Gradient Boosting Classifier
This data science in python project predicts if a loan should be given to an applicant or not. We predict if the customer is eligible for loan based on several factors like credit score and past history.