How to do recursive feature elimination in Python (DecisionTreeRegressor)?
FEATURE EXTRACTION DATA CLEANING PYTHON DATA MUNGING MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to do recursive feature elimination in Python (DecisionTreeRegressor)?

How to do recursive feature elimination in Python (DecisionTreeRegressor)?

This recipe helps you do recursive feature elimination in Python (DecisionTreeRegressor)

0

Recipe Objective

While training a model there are many feature which are recursive and it effects the computational cost.

So this is the recipe on how we can do recursive feature elimination in Python (DecisionTreeRegressor).

Step 1 - Importing Library

from sklearn.datasets import make_regression from sklearn.feature_selection import RFECV from sklearn.tree import DecisionTreeRegressor

We have only imported make_regression, RFECV, DecisionTreeRegressor which is needed.

Step 2 - Creating dataset

We have created a dataset by function make_regression with desired parameters. X, y = make_regression(n_samples = 10000, n_features = 100, n_informative = 2) print(X.shape)

Step 3 - Using recursive feature eliminator

.

  • We are using DecisionTreeRegressor as a model
  • dtree = DecisionTreeRegressor()
  • Using Recursive feature elemination. We have passed the required parameters as model, step, scoring, cv and verbose. We have fited the data and transformed it to remove the recursive feature.
  • rfecv = RFECV(estimator=dtree, step=1, scoring="neg_mean_squared_error", cv=4, verbose=1, n_jobs = 4) rfecv.fit(X, y) rfecv.transform(X) print(rfecv) print(rfecv.n_features_) So the output comes as:

    (10000, 100)
    Fitting estimator with 100 features.
    Fitting estimator with 99 features.
    Fitting estimator with 98 features.
    Fitting estimator with 97 features.
    Fitting estimator with 96 features.
    Fitting estimator with 95 features.
    Fitting estimator with 94 features.
    Fitting estimator with 93 features.
    Fitting estimator with 92 features.
    Fitting estimator with 91 features.
    Fitting estimator with 90 features.
    Fitting estimator with 89 features.
    Fitting estimator with 88 features.
    Fitting estimator with 87 features.
    Fitting estimator with 86 features.
    Fitting estimator with 85 features.
    Fitting estimator with 84 features.
    Fitting estimator with 83 features.
    Fitting estimator with 82 features.
    Fitting estimator with 81 features.
    Fitting estimator with 80 features.
    Fitting estimator with 79 features.
    Fitting estimator with 78 features.
    Fitting estimator with 77 features.
    Fitting estimator with 76 features.
    Fitting estimator with 75 features.
    Fitting estimator with 74 features.
    Fitting estimator with 73 features.
    Fitting estimator with 72 features.
    Fitting estimator with 71 features.
    Fitting estimator with 70 features.
    Fitting estimator with 69 features.
    Fitting estimator with 68 features.
    Fitting estimator with 67 features.
    Fitting estimator with 66 features.
    Fitting estimator with 65 features.
    Fitting estimator with 64 features.
    Fitting estimator with 63 features.
    Fitting estimator with 62 features.
    Fitting estimator with 61 features.
    Fitting estimator with 60 features.
    Fitting estimator with 59 features.
    Fitting estimator with 58 features.
    Fitting estimator with 57 features.
    Fitting estimator with 56 features.
    Fitting estimator with 55 features.
    Fitting estimator with 54 features.
    Fitting estimator with 53 features.
    Fitting estimator with 52 features.
    Fitting estimator with 51 features.
    Fitting estimator with 50 features.
    Fitting estimator with 49 features.
    Fitting estimator with 48 features.
    Fitting estimator with 47 features.
    Fitting estimator with 46 features.
    Fitting estimator with 45 features.
    Fitting estimator with 44 features.
    Fitting estimator with 43 features.
    Fitting estimator with 42 features.
    Fitting estimator with 41 features.
    Fitting estimator with 40 features.
    Fitting estimator with 39 features.
    Fitting estimator with 38 features.
    Fitting estimator with 37 features.
    Fitting estimator with 36 features.
    Fitting estimator with 35 features.
    Fitting estimator with 34 features.
    Fitting estimator with 33 features.
    Fitting estimator with 32 features.
    Fitting estimator with 31 features.
    Fitting estimator with 30 features.
    Fitting estimator with 29 features.
    Fitting estimator with 28 features.
    Fitting estimator with 27 features.
    Fitting estimator with 26 features.
    Fitting estimator with 25 features.
    Fitting estimator with 24 features.
    Fitting estimator with 23 features.
    Fitting estimator with 22 features.
    Fitting estimator with 21 features.
    Fitting estimator with 20 features.
    Fitting estimator with 19 features.
    Fitting estimator with 18 features.
    Fitting estimator with 17 features.
    Fitting estimator with 16 features.
    Fitting estimator with 15 features.
    Fitting estimator with 14 features.
    Fitting estimator with 13 features.
    Fitting estimator with 12 features.
    Fitting estimator with 11 features.
    Fitting estimator with 10 features.
    Fitting estimator with 9 features.
    Fitting estimator with 8 features.
    Fitting estimator with 7 features.
    Fitting estimator with 6 features.
    Fitting estimator with 5 features.
    Fitting estimator with 4 features.
    Fitting estimator with 3 features.
    
    RFECV(cv=4,
       estimator=DecisionTreeRegressor(criterion="mse", max_depth=None, max_features=None,
               max_leaf_nodes=None, min_impurity_decrease=0.0,
               min_impurity_split=None, min_samples_leaf=1,
               min_samples_split=2, min_weight_fraction_leaf=0.0,
               presort=False, random_state=None, splitter="best"),
       min_features_to_select=1, n_jobs=4, scoring="neg_mean_squared_error",
       step=1, verbose=1)
    
    2
    
    

Relevant Projects

Predict Credit Default | Give Me Some Credit Kaggle
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Predict Census Income using Deep Learning Models
In this project, we are going to work on Deep Learning using H2O to predict Census income.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Credit Card Fraud Detection as a Classification Problem
In this data science project, we will predict the credit card fraud in the transactional dataset using some of the predictive models.

Deep Learning with Keras in R to Predict Customer Churn
In this deep learning project, we will predict customer churn using Artificial Neural Networks and learn how to model an ANN in R with the keras deep learning package.

Demand prediction of driver availability using multistep time series analysis
In this supervised learning machine learning project, you will predict the availability of a driver in a specific area by using multi step time series analysis.

Time Series Forecasting with LSTM Neural Network Python
Deep Learning Project- Learn to apply deep learning paradigm to forecast univariate time series data.

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.

Build a Collaborative Filtering Recommender System in Python
Use the Amazon Reviews/Ratings dataset of 2 Million records to build a recommender system using memory-based collaborative filtering in Python.