What are the updates functions in lasagne?

This recipe explains what are the updates functions in lasagne.

Recipe Objective - What are the updates functions in lasagne?

Lasagne provides the "lasagne.updates" class for update function.

The Update functions are used to generate Theano update dictionaries for training.

This function implements different methods to control the learning rate.

Update Functions:-

sgd - Stochastic Gradient Descent (SGD) updates.

momentum - Stochastic Gradient Descent (SGD) updates with momentum.

nesterov_momentum - Stochastic Gradient Descent (SGD) updates with Nesterov momentum.

adagrad - Adagrad updates.

rmsprop - RMSProp updates.

adadelta - Adadelta updates.

adam - Adam updates.

adamax - Adamax updates.

amsgrad - AMSGrad updates.

Links for the more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/neural-network-projects

Update Functions Example:-

import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.updates import nesterov_momentum
ly_in = InputLayer((100, 20))
ly1 = DenseLayer(ly_in, num_units=3, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(ly1, x)
params = lasagne.layers.get_all_params(ly1)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
updates = nesterov_momentum(loss, params, learning_rate=1e-4, momentum=.9)
train_fn = theano.function([x, y], updates=updates)

updates = lasagne.updates.rmsprop(loss, params, learning_rate=0.0001)
updates = lasagne.updates.apply_momentum(updates, params, momentum=0.9)
updates

OrderedDict([(<TensorType(float64, matrix)>, Elemwise{add,no_inplace}.0),
             (W, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{add,no_inplace}.0),
             (b, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, matrix)>, Elemwise{sub,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

What Users are saying..

profile image

Gautam Vermani

Data Consultant at Confidential
linkedin profile url

Having worked in the field of Data Science, I wanted to explore how I can implement projects in other domains, So I thought of connecting with ProjectPro. A project that helped me absorb this topic... Read More

Relevant Projects

Build a Multi ClassText Classification Model using Naive Bayes
Implement the Naive Bayes Algorithm to build a multi class text classification model in Python.

Build Real Estate Price Prediction Model with NLP and FastAPI
In this Real Estate Price Prediction Project, you will learn to build a real estate price prediction machine learning model and deploy it on Heroku using FastAPI Framework.

Topic modelling using Kmeans clustering to group customer reviews
In this Kmeans clustering machine learning project, you will perform topic modelling in order to group customer reviews based on recurring patterns.

End-to-End Snowflake Healthcare Analytics Project on AWS-1
In this Snowflake Healthcare Analytics Project, you will leverage Snowflake on AWS to predict patient length of stay (LOS) in hospitals. The prediction of LOS can help in efficient resource allocation, lower the risk of staff/visitor infections, and improve overall hospital functioning.

Census Income Data Set Project-Predict Adult Census Income
Use the Adult Income dataset to predict whether income exceeds 50K yr based oncensus data.

Predictive Analytics Project for Working Capital Optimization
In this Predictive Analytics Project, you will build a model to accurately forecast the timing of customer and supplier payments for optimizing working capital.

Build a Graph Based Recommendation System in Python -Part 1
Python Recommender Systems Project - Learn to build a graph based recommendation system in eCommerce to recommend products.

Learn to Build a Polynomial Regression Model from Scratch
In this Machine Learning Regression project, you will learn to build a polynomial regression model to predict points scored by the sports team.

Stock Price Prediction Project using LSTM and RNN
Learn how to predict stock prices using RNN and LSTM models. Understand deep learning concepts and apply them to real-world financial data for accurate forecasting.

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction