How does automatic differentiation work in MXNet?

In this recipe, we will see how does automatic differentiation work in MXNet with an example

Recipe Objective: How does automatic differentiation work in MXNet? Explain with an example

This recipe explains how does automatic differentiation work in MXNet.

Access Retail Price Recommendation ML Project with Source Code

Step 1: Importing library

Let us first import the necessary libraries. We'll import mxnet, autograd, and ndarray (nd as short alias) packages.

import mxnet as mx
from mxnet import nd
from mxnet import autograd

Step 2: Basic functionality

For example, let's assume we want to differentiate a function f(a)=(4b^2)+(2*b) with respect to a. Now, we'll store the gradient of f(a) by invoking its attach_grad method. After that, we'll define f(a)=(4b^2)+(2*b) inside a autograd.record() to allow MXNet store y. By invoking b.backward(), we'll call backdrop. Now well cross-check expected output and computed output.

a = nd.array([[1,2,3], [3,4,5], [6,7,8]])
a.attach_grad()
with autograd.record():
    b = (4*a*a) + (2*a)
b.backward()
a.grad

Step 3: Dynamic Usage

For a dynamic program computation that depends on real-time values, MXNet will save and compute the gradient. The function will compute the square of input until the norm reaches 2000. We'll print the gradient of the function.

x = nd.array([[1,2,3], [3,4,5], [6,7,8]])
x.attach_grad()
with autograd.record():
    y = x * x
    while y.norm().asscalar() < 2000:
       y = y * y
    if y.sum().asscalar() >= 0:
       z = y[0]
    else:
       z = y[1]
z.backward
x.grad

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Learn to Build an End-to-End Machine Learning Pipeline - Part 2
In this Machine Learning Project, you will learn how to build an end-to-end machine learning pipeline for predicting truck delays, incorporating Hopsworks' feature store and Weights and Biases for model experimentation.

Build a Graph Based Recommendation System in Python-Part 2
In this Graph Based Recommender System Project, you will build a recommender system project for eCommerce platforms and learn to use FAISS for efficient similarity search.

End-to-End ML Model Monitoring using Airflow and Docker
In this MLOps Project, you will learn to build an end to end pipeline to monitor any changes in the predictive power of model or degradation of data.

Recommender System Machine Learning Project for Beginners-4
Collaborative Filtering Recommender System Project - Comparison of different model based and memory based methods to build recommendation system using collaborative filtering.

Deploying Machine Learning Models with Flask for Beginners
In this MLOps on GCP project you will learn to deploy a sales forecasting ML Model using Flask.

Isolation Forest Model and LOF for Anomaly Detection in Python
Credit Card Fraud Detection Project - Build an Isolation Forest Model and Local Outlier Factor (LOF) in Python to identify fraudulent credit card transactions.

Build Real Estate Price Prediction Model with NLP and FastAPI
In this Real Estate Price Prediction Project, you will learn to build a real estate price prediction machine learning model and deploy it on Heroku using FastAPI Framework.

Image Segmentation using Mask R-CNN with Tensorflow
In this Deep Learning Project on Image Segmentation Python, you will learn how to implement the Mask R-CNN model for early fire detection.

Build a Face Recognition System in Python using FaceNet
In this deep learning project, you will build your own face recognition system in Python using OpenCV and FaceNet by extracting features from an image of a person's face.

Recommender System Machine Learning Project for Beginners-1
Recommender System Machine Learning Project for Beginners - Learn how to design, implement and train a rule-based recommender system in Python