What is the difference between softmax and logsoftmax in pytorch

This recipe explains what is the difference between softmax and logsoftmax in pytorch

Recipe Objective

What is the difference between softmax and logsoftmax in pytorch?

In case of the Softmax Function, it is applied to an n-dim input tensor in which it will be rescaling them so that the elements of the output n-dim tensor lie in the range [0,1] and sum to 1. The function will return the similar shape and dimension as the input with the values in range[0,1]. The Softmax function is defined as:
Softmax(xi)= exp(xi) / ∑ j exp(xj)
In the case of Logsoftmax function which is nothing but the log of Softmax function. It will return the same shape and dimension as the input with the values in the range [-inf, 0]. The Logsoftmax function is defined as:
LogSoftmax(xi) = log (exp(xi) / ∑ j exp(xj))

Step 1 - Import library

import torch

Step 2 - Softmax function

softmax = torch.nn.Softmax(dim=1)
tensor_input = torch.randn(3,4)
print("The output for the softmax function is:","\n",softmax(tensor_input))

The output for the softmax function is: 
 tensor([[0.1426, 0.2480, 0.4632, 0.1463],
        [0.2917, 0.1799, 0.2977, 0.2307],
        [0.1568, 0.1523, 0.2969, 0.3941]])

Step 3 - LogSoftmax function

Logsoftmax = torch.nn.LogSoftmax()
tensor_input2 = torch.randn(3,4)
print("The output for the LogSoftmax function is:","\n",Logsoftmax(tensor_input2))

The output for the LogSoftmax function is: 
 tensor([[-0.9011, -1.3806, -2.2884, -1.4228],
        [-0.9425, -1.4145, -2.0483, -1.4340],
        [-0.8965, -2.5074, -2.0916, -0.9493]])
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:3: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  This is separate from the ipykernel package so we can avoid doing imports until

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

BigMart Sales Prediction ML Project in Python
The goal of the BigMart Sales Prediction ML project is to build and evaluate different predictive models and determine the sales of each product at a store.

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction

Topic modelling using Kmeans clustering to group customer reviews
In this Kmeans clustering machine learning project, you will perform topic modelling in order to group customer reviews based on recurring patterns.

Time Series Forecasting Project-Building ARIMA Model in Python
Build a time series ARIMA model in Python to forecast the use of arrival rate density to support staffing decisions at call centres.

End-to-End Snowflake Healthcare Analytics Project on AWS-1
In this Snowflake Healthcare Analytics Project, you will leverage Snowflake on AWS to predict patient length of stay (LOS) in hospitals. The prediction of LOS can help in efficient resource allocation, lower the risk of staff/visitor infections, and improve overall hospital functioning.

NLP and Deep Learning For Fake News Classification in Python
In this project you will use Python to implement various machine learning methods( RNN, LSTM, GRU) for fake news classification.

Loan Eligibility Prediction in Python using H2O.ai
In this loan prediction project you will build predictive models in Python using H2O.ai to predict if an applicant is able to repay the loan or not.

Deep Learning Project for Text Detection in Images using Python
CV2 Text Detection Code for Images using Python -Build a CRNN deep learning model to predict the single-line text in a given image.

NLP Project on LDA Topic Modelling Python using RACE Dataset
Use the RACE dataset to extract a dominant topic from each document and perform LDA topic modeling in python.