How to tune Hyper parameters using Random Search in Python?

This recipe helps you tune Hyper parameters using Random Search in Python

Recipe Objective

Many a times while working on a dataset and using a Machine Learning model we don"t know which set of hyperparameters will give us the best result. Passing all sets of hyperparameters manually through the model and checking the result might be a hectic work and may not be possible to do.

To get the best set of hyperparameters we can use Grid Search. Random Search passes Random combinations of hyperparameters one by one into the model and check the result. Finally it gives us the set of hyperparemeters which gives the best result after passing in the model.

So this recipe is a short example of how can tune Hyper-parameters using Random Search in Python

Access Face Recognition Project Code using Facenet in Python

Step 1 - Import the library - RandomizedSearchCv

from scipy.stats import uniform from sklearn import linear_model, datasets from sklearn.model_selection import RandomizedSearchCV

Here we have imported various modules like datasets, uniform, linear_model and RandomizedSearchCV from differnt libraries. We will understand the use of these later while using it in the in the code snipet.
For now just have a look on these imports.

Step 2 - Setup the Data

Here we have used datasets to load the inbuilt iris dataset and we have created objects X and y to store the data and the target value respectively. iris = datasets.load_iris() X = iris.data y = iris.target

Step 3 - Using Model

Here, we are using Logistic Regression as a Machine Learning model to use RandomisedSearchCV. So we have created an object Logistic. logistic = linear_model.LogisticRegression()

Step 5 - Parameters to be optimized

Logistic Regression requires two parameters "C" and "penalty" to be optimised by RandomisedSearchCV. So we have set these two parameters as a list of values form which RandomisedSearchCV will select the best value of parameter. C = uniform(loc=0, scale=4) penalty = ["l1", "l2"] hyperparameters = dict(C=C, penalty=penalty)

Step 6 - Using RandomisedSearchCV and Printing Results

Before using RandomisedSearchCV, lets have a look on the important parameters.

  • estimator: In this we have to pass the models or functions on which we want to use RandomisedSearchCV
  • param_grid: Dictionary or list of parameters of models or function in which RandomisedSearchCV have to select the best.
  • Scoring: It is used as a evaluating metric for the model performance to decide the best hyperparameters, if not especified then it uses estimator score.

Making an object clf for RandomisedSearchCV and fitting the dataset i.e X and y clf = RandomizedSearchCV(logistic, hyperparameters, random_state=1, n_iter=100, cv=5, verbose=0, n_jobs=-1) best_model = clf.fit(X, y) Now we are using print statements to print the results. It will give the values of hyperparameters as a result. print("Best Penalty:", best_model.best_estimator_.get_params()["penalty"]) print("Best C:", best_model.best_estimator_.get_params()["C"]) As an output we get:

Best Penalty: l1
Best C: 1.668088018810296

Download Materials

What Users are saying..

profile image

Gautam Vermani

Data Consultant at Confidential
linkedin profile url

Having worked in the field of Data Science, I wanted to explore how I can implement projects in other domains, So I thought of connecting with ProjectPro. A project that helped me absorb this topic... Read More

Relevant Projects

BERT Text Classification using DistilBERT and ALBERT Models
This Project Explains how to perform Text Classification using ALBERT and DistilBERT

End-to-End Snowflake Healthcare Analytics Project on AWS-2
In this AWS Snowflake project, you will build an end to end retraining pipeline by checking Data and Model Drift and learn how to redeploy the model if needed

Build a Graph Based Recommendation System in Python-Part 2
In this Graph Based Recommender System Project, you will build a recommender system project for eCommerce platforms and learn to use FAISS for efficient similarity search.

Predict Churn for a Telecom company using Logistic Regression
Machine Learning Project in R- Predict the customer churn of telecom sector and find out the key drivers that lead to churn. Learn how the logistic regression model using R can be used to identify the customer churn in telecom dataset.

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

Learn Hyperparameter Tuning for Neural Networks with PyTorch
In this Deep Learning Project, you will learn how to optimally tune the hyperparameters (learning rate, epochs, dropout, early stopping) of a neural network model in PyTorch to improve model performance.

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .

AWS MLOps Project to Deploy Multiple Linear Regression Model
Build and Deploy a Multiple Linear Regression Model in Python on AWS

Deep Learning Project- Real-Time Fruit Detection using YOLOv4
In this deep learning project, you will learn to build an accurate, fast, and reliable real-time fruit detection system using the YOLOv4 object detection model for robotic harvesting platforms.

Mastering A/B Testing: A Practical Guide for Production
In this A/B Testing for Machine Learning Project, you will gain hands-on experience in conducting A/B tests, analyzing statistical significance, and understanding the challenges of building a solution for A/B testing in a production environment.