How to optimize a function using Adam in R

This recipe helps you optimize a function using Adam in R

Recipe Objective -How to optimize a function using Adam in R?

The Adam or Adaptive Moment Estimation optimizer is defined as an optimizer that computes adaptive learning rates for each parameter. The Adam optimizer in addition of storing an exponentially decaying average of the past squared gradients defined as vt, like the Adadelta optimizer and the RMSprop optimizer, Adam optimizer keeps an exponentially decaying average of the past gradients defined as mt, which is similar to the momentum. The Adam optimizer behaves like a heavy ball with the friction which prefers flat minima in a error surface. The Adam optimizer can handle the sparse gradients on the noisy problems. The Adam optimizer is relatively easy to configure in which the default configuration parameters does well on most of the problems.

This recipe explains what is Adam optimizer, what are its benefits and how it can be executed.

MNIST Handwritten Digit Classification using Machine Learning

Implementing Adam optimizer.

Step 1: Installing and Loading keras package to build neural network using keras.

# Installing Packages
install.packages("keras")
# Loading packages
library(keras)

Step 2: Loading MNIST handwritten digit dataset which comes pre-loaded in keras package.

# Loading the data
mnist <- dataset_mnist()

Step 3: Train and test dataset containing images are prepared using MNIST dataset.

# Preparing train data and test data
training_images <- mnist$train$x
training_labels <- mnist$train$y
testing_images <- mnist$test$x
testing_labels <- mnist$test$y

Step 4: Transform the train images data and test image data into the double array of [0, 255] shape (60000, 28 * 28) with values between 0 and 1.

# Reshaping train data and test data
training_images <- array_reshape(training_images, c(60000, 28 * 28))
training_images <- training_images / 255
testing_images <- array_reshape(testing_images, c(10000, 28 * 28))
testing_images <- testing_images / 255

Step 5: Labels are prepared by categorically encoding them.

# Preparing Labels
training_labels <- to_categorical(training_labels)
testing_labels <- to_categorical(testing_labels)

Step 6: Model is build using dense layers with relu and softmax activation.

# Model Buidling
neural_network <- keras_model_sequential() %>%
layer_dense(units = 512, activation = "relu", input_shape = c(28 * 28)) %>%
layer_dense(units = 10, activation = "softmax")

Step 7: Model is compiled with optimizer Adam, loss as categorical entropy and accuracy as metrics.

# Model Compiling
neural_network %>% compile(
optimizer = "adam",
loss = "categorical_crossentropy",
metrics = c("accuracy")

Step 8: Model is fitted. neural_network %>% fit(training_images, training_labels, epochs = 3, batch_size = 64)

Step 9: Model performance is evaluated on testing dataset of images and test dataset of labels.

# Model performance
metric <- neural_network %>% evaluate(testing_images, testing_labels)

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Deep Learning Project for Text Detection in Images using Python
CV2 Text Detection Code for Images using Python -Build a CRNN deep learning model to predict the single-line text in a given image.

GCP MLOps Project to Deploy ARIMA Model using uWSGI Flask
Build an end-to-end MLOps Pipeline to deploy a Time Series ARIMA Model on GCP using uWSGI and Flask

Expedia Hotel Recommendations Data Science Project
In this data science project, you will contextualize customer data and predict the likelihood a customer will stay at 100 different hotel groups.

Multi-Class Text Classification with Deep Learning using BERT
In this deep learning project, you will implement one of the most popular state of the art Transformer models, BERT for Multi-Class Text Classification

Hands-On Approach to Causal Inference in Machine Learning
In this Machine Learning Project, you will learn to implement various causal inference techniques in Python to determine, how effective the sprinkler is in making the grass wet.

MLOps AWS Project on Topic Modeling using Gunicorn Flask
In this project we will see the end-to-end machine learning development process to design, build and manage reproducible, testable, and evolvable machine learning models by using AWS

Learn to Build a Neural network from Scratch using NumPy
In this deep learning project, you will learn to build a neural network from scratch using NumPy

Build Customer Propensity to Purchase Model in Python
In this machine learning project, you will learn to build a machine learning model to estimate customer propensity to purchase.

Credit Card Default Prediction using Machine learning techniques
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.