What is Gradient descent explain with example in R

This recipe explains what is Gradient descent explain with example in R

Recipe Objective - What is Gradient Descent. Explain with Example in R?

Gradient descent in neural network is an optimization algorithm that is used to find values of parameters or coefficients of the function that minimizes the cost function. The gradient descent is also known as engine of the neural network. Gradient descent is mostly used when the parameters cannot be calculated analytically that is in linear algebra and must be searched for by optimization algorithm. Gradient descent is of three types: Batch gradient descent, stochastic gradient descent and mini-batch gradient descent. Batch gradient descent refers to calculating derivative from all training data before calculating an update. Stochastic gradient descent refers to calculating derivative from each training data instance and calculating the update immediately. Linear regression is an example of gradient descent as in y = mx + c, where "y" is a response or outcome variable, "m" is a gradient of linear trend-line, "x" is a predictor variable and "c" is an intercept where an intercept is a point on the y axis where a value of the predictor x is always zero.

This recipe explains what is Gradient Descent, what are its types and how it can be executed.

Implementing Gradient Descent.

Step 1: Packages are installed and loaded to create model displaying gradient descent.

# Installing Packages
install.packages("e1071")
install.packages("caTools")
install.packages("caret")
# Loading package
library(e1071)
library(caTools)
library(caret)

Step 2: mtcars dataset which comes preloaded in R is attached for using gradient descent.

# Loading data
attach(mtcars)

Step 3: Linear regression model is build predicting mpg(mileage) using hp(horse power) feature using mtcars dataset. Appropriate values for the intercept and slope are found using lm() function which creates the linear model using which most appropriate intercept and slope or coefficients are found.

# Model Building
model_grad <- lm(mpg ~ hp, data = mtcars)

Step 4: Coefficients in the model are found using coef() function.

# Coefficients in model
coef(model_grad)
(Intercept) hp 30.09886054 -0.06822828 # The intercept is set at 30.09886054 on y axis and that the gradient is set at -0.06822828. The negative gradient
# tells that there is an inverse relationship between mpg and horse power with the one unit increase in horse power results in a
# 0.06 unit decrease in mpg that is mileage.

Step 5: Predictions are done on model using predict() function.

# Model prediction
y_pred <- predict(model_grad)

Further more, plot of model can be made using abline() function. Mean square error or mse can also be calculated by summing the squared differences between the observed y value and the predicted y value and then dividing by number of observations that is n.

What Users are saying..

profile image

Ameeruddin Mohammed

ETL (Abintio) developer at IBM
linkedin profile url

I come from a background in Marketing and Analytics and when I developed an interest in Machine Learning algorithms, I did multiple in-class courses from reputed institutions though I got good... Read More

Relevant Projects

NLP Project for Beginners on Text Processing and Classification
This Project Explains the Basic Text Preprocessing and How to Build a Classification Model in Python

Topic modelling using Kmeans clustering to group customer reviews
In this Kmeans clustering machine learning project, you will perform topic modelling in order to group customer reviews based on recurring patterns.

House Price Prediction Project using Machine Learning in Python
Use the Zillow Zestimate Dataset to build a machine learning model for house price prediction.

Build a Logistic Regression Model in Python from Scratch
Regression project to implement logistic regression in python from scratch on streaming app data.

Insurance Pricing Forecast Using XGBoost Regressor
In this project, we are going to talk about insurance forecast by using linear and xgboost regression techniques.

Loan Eligibility Prediction Project using Machine learning on GCP
Loan Eligibility Prediction Project - Use SQL and Python to build a predictive model on GCP to determine whether an application requesting loan is eligible or not.

Expedia Hotel Recommendations Data Science Project
In this data science project, you will contextualize customer data and predict the likelihood a customer will stay at 100 different hotel groups.

Time Series Forecasting Project-Building ARIMA Model in Python
Build a time series ARIMA model in Python to forecast the use of arrival rate density to support staffing decisions at call centres.

NLP and Deep Learning For Fake News Classification in Python
In this project you will use Python to implement various machine learning methods( RNN, LSTM, GRU) for fake news classification.

Build Portfolio Optimization Machine Learning Models in R
Machine Learning Project for Financial Risk Modelling and Portfolio Optimization with R- Build a machine learning model in R to develop a strategy for building a portfolio for maximized returns.