What Does Torch No Grad Do?

This beginner-friendly Pytorch code shows you how to implement the torch.no_grad() function.

Objective: What Does Torch No Grad Do?

This PyTorch code example will teach you to use the ‘torch.no_grad()’ function for a PyTorch tensor. 

What Does PyTorch Torch.No_Grad() Do?

The PyTorch torch.no_grad() function is a context manager that disables gradient calculation. This is useful for inference when you are sure you will not call tensor.backward(). It will reduce memory consumption for computations that would otherwise have requires_grad=True. In this mode, the result of every computation will have requires_grad=False, even when the inputs have requires_grad=True. One exception- factory functions, or functions that create a new Tensor and take a requires_grad kwarg, will not be affected by this mode.

Steps Showing How To Use Torch Tensor No_Grad() in PyTorch

The following steps will show you how to use the torch.no_grad() function to disable the gradient calculation in an easy-to-understand PyTorch tensor example.

Step 1 - Import Library For Using torch.no_grad()

First, you must import the required libraries.

import torch

Step 2 - Take Sample PyTorch Tensor

The next step is to take any sample tensor for the example.

tensor = torch.tensor([4.], requires_grad=True)

Get Closer To Your Dream of Becoming a Data Scientist with Solved End-to-End PyTorch Projects

Step 3 - Using ‘with torch.no_grad()’ in PyTorch

The final step is to disable gradient calculation using the ‘with torch.no_grad()’ function.

with torch.no_grad():

  output = tensor * 3

output.requires_grad

The output of the above code is-

False

Example Of torch.no_grad() For Loop

The torch.no_grad() context manager can be used in a for loop to disable gradient calculation for all operations within the loop. 

Here is an example of how to use torch.no_grad() in a for loop-

import torch

def predict_batch(model, inputs):

  with torch.no_grad():

    outputs = []

    for input in inputs:

      output = model(input)

      outputs.append(output)

  return outputs

# Load a pre-trained model

model = torch.load('model.pt')

# Make a prediction on a batch of inputs

inputs = torch.rand((10, 3))

outputs = predict_batch(model, inputs)

# Print the predictions

for output in outputs:

  print(output)

The output of the above code-

tensor([1.2345])

tensor([0.9876])

tensor([0.5432])

...

Torch.no_grad() Decorator Example

You can use the torch.no_grad() decorator by simply wrapping the function or block of code you want to disable gradient calculation for in the decorator. For example-

import torch

@torch.no_grad()

def predict(model, input):

  output = model(input)

  return output

# Load a pre-trained model

model = torch.load('model.pt')

# Make a prediction

input = torch.tensor([1.])

output = predict(model, input)

# Print the prediction

print(output)

The output of the above code is-

tensor([3.])

In the above example, the predict() function is decorated with torch.no_grad(). This means that gradient calculation will be disabled for all operations within the function, even though the input tensor has requires_grad=True.

Explore The Real-World Implementations of Torch.No_Grad() With ProjectPro

This PyTorch code example has discussed the importance of torch.no_grad() in PyTorch, explaining its role in disabling gradient computation and the steps to utilize it effectively in various scenarios. We have covered its standard usage and as a decorator for various other functions. You can further deepen your PyTorch expertise and apply it to real-world data science and machine learning projects by exploring PyTorch projects offered by ProjectPro. These projects, guided by experts, provide hands-on experience and practical insights to solve complex data science challenges. By engaging in these enterprise-grade projects from the ProjectPro repository, you can build the skills and confidence needed to excel in data science and machine learning.

What Users are saying..

profile image

Ameeruddin Mohammed

ETL (Abintio) developer at IBM
linkedin profile url

I come from a background in Marketing and Analytics and when I developed an interest in Machine Learning algorithms, I did multiple in-class courses from reputed institutions though I got good... Read More

Relevant Projects

Loan Eligibility Prediction in Python using H2O.ai
In this loan prediction project you will build predictive models in Python using H2O.ai to predict if an applicant is able to repay the loan or not.

Insurance Pricing Forecast Using XGBoost Regressor
In this project, we are going to talk about insurance forecast by using linear and xgboost regression techniques.

GCP MLOps Project to Deploy ARIMA Model using uWSGI Flask
Build an end-to-end MLOps Pipeline to deploy a Time Series ARIMA Model on GCP using uWSGI and Flask

MLOps using Azure Devops to Deploy a Classification Model
In this MLOps Azure project, you will learn how to deploy a classification machine learning model to predict the customer's license status on Azure through scalable CI/CD ML pipelines.

PyTorch Project to Build a GAN Model on MNIST Dataset
In this deep learning project, you will learn how to build a GAN Model on MNIST Dataset for generating new images of handwritten digits.

NLP Project to Build a Resume Parser in Python using Spacy
Use the popular Spacy NLP python library for OCR and text classification to build a Resume Parser in Python.

Build Classification Algorithms for Digital Transformation[Banking]
Implement a machine learning approach using various classification techniques in Python to examine the digitalisation process of bank customers.

Expedia Hotel Recommendations Data Science Project
In this data science project, you will contextualize customer data and predict the likelihood a customer will stay at 100 different hotel groups.

AWS Project to Build and Deploy LSTM Model with Sagemaker
In this AWS Sagemaker Project, you will learn to build a LSTM model on Sagemaker for sales forecasting while analyzing the impact of weather conditions on Sales.

End-to-End Speech Emotion Recognition Project using ANN
Speech Emotion Recognition using RAVDESS Audio Dataset - Build an Artificial Neural Network Model to Classify Audio Data into various Emotions like Sad, Happy, Angry, and Neutral