How to optimize a function using SGD in R

This recipe helps you optimize a function using SGD in R

Recipe Objective -How to optimize a function using SGD in R?

The SGD or Stochastic gradient descent SGD is defined as a variant of gradient descent in which instead of performing computations on whole dataset which is mostly redundant and inefficient, Stochastic Gradient Descent only computes on the small subset or the random selection of the data examples. The stochastic gradient descent produces the same performance as a regular gradient descent whenever the learning rate is low. The stochastic gradient descent performs one update at a time and thus it is usually much faster and also used to learn online. The Stochastic Gradient Descent performs the frequent updates with a high variance which causes the objective function to fluctuate heavily that is at much faster rate. The stochastic gradient descent enables itself to jump to the new and potentially better local minima as it keeps overshooting.

This recipe explains what is SGD optimizer, what are its benefits and how it can be excecuted.

Implementing SGD optimizer.

Step 1: Installing and Loading keras package to build neural network using keras.

# Installing Packages
install.packages("keras")
# Loading packages
library(keras)

Step 2: Loading MNIST handwritten digit dataset which comes pre-loaded in keras package.

# Loading the data
mnist <- dataset_mnist()

Step 3: Train and test dataset containing images are prepared using MNIST dataset.

# Preparing train data and test data
training_images <- mnist$train$x
training_labels <- mnist$train$y
testing_images <- mnist$test$x
testing_labels <- mnist$test$y

Step 4: Transform the train images data and test image data into the double array of [0, 255] shape (60000, 28 * 28) with values between 0 and 1.

# Reshaping train data and test data
training_images <- array_reshape(training_images, c(60000, 28 * 28))
training_images <- training_images / 255
testing_images <- array_reshape(testing_images, c(10000, 28 * 28))
testing_images <- testing_images / 255

Step 5: Labels are prepared by categorically encoding them.

# Preparing Labels
training_labels <- to_categorical(training_labels)
testing_labels <- to_categorical(testing_labels)

Step 6: Model is build using dense layers with relu and softmax activation.

# Model Buidling
neural_network <- keras_model_sequential() %>%
layer_dense(units = 512, activation = "relu", input_shape = c(28 * 28)) %>%
layer_dense(units = 10, activation = "softmax")

Step 7: Model is compiled with optimizer SGD, loss as categorical entropy and accuracy as metrics.

# Model Compiling
neural_network %>% compile(
optimizer = "sgd",
loss = "categorical_crossentropy",
metrics = c("accuracy")

Step 8: Model is fitted. neural_network %>% fit(training_images, training_labels, epochs = 3, batch_size = 64)

Step 9: Model performance is evaluated on testing dataset of images and test dataset of labels.

# Model performance
metric <- neural_network %>% evaluate(testing_images, testing_labels)

What Users are saying..

profile image

Anand Kumpatla

Sr Data Scientist @ Doubleslash Software Solutions Pvt Ltd
linkedin profile url

ProjectPro is a unique platform and helps many people in the industry to solve real-life problems with a step-by-step walkthrough of projects. A platform with some fantastic resources to gain... Read More

Relevant Projects

PyTorch Project to Build a LSTM Text Classification Model
In this PyTorch Project you will learn how to build an LSTM Text Classification model for Classifying the Reviews of an App .

Build CNN for Image Colorization using Deep Transfer Learning
Image Processing Project -Train a model for colorization to make grayscale images colorful using convolutional autoencoders.

Loan Eligibility Prediction in Python using H2O.ai
In this loan prediction project you will build predictive models in Python using H2O.ai to predict if an applicant is able to repay the loan or not.

Deep Learning Project- Real-Time Fruit Detection using YOLOv4
In this deep learning project, you will learn to build an accurate, fast, and reliable real-time fruit detection system using the YOLOv4 object detection model for robotic harvesting platforms.

Classification Projects on Machine Learning for Beginners - 2
Learn to implement various ensemble techniques to predict license status for a given business.

Recommender System Machine Learning Project for Beginners-4
Collaborative Filtering Recommender System Project - Comparison of different model based and memory based methods to build recommendation system using collaborative filtering.

Build Real Estate Price Prediction Model with NLP and FastAPI
In this Real Estate Price Prediction Project, you will learn to build a real estate price prediction machine learning model and deploy it on Heroku using FastAPI Framework.

Build CI/CD Pipeline for Machine Learning Projects using Jenkins
In this project, you will learn how to create a CI/CD pipeline for a search engine application using Jenkins.

Predict Churn for a Telecom company using Logistic Regression
Machine Learning Project in R- Predict the customer churn of telecom sector and find out the key drivers that lead to churn. Learn how the logistic regression model using R can be used to identify the customer churn in telecom dataset.

Llama2 Project for MetaData Generation using FAISS and RAGs
In this LLM Llama2 Project, you will automate metadata generation using Llama2, RAGs, and AWS to reduce manual efforts.