What are the updates functions in lasagne?

This recipe explains what are the updates functions in lasagne.

Recipe Objective - What are the updates functions in lasagne?

Lasagne provides the "lasagne.updates" class for update function.

The Update functions are used to generate Theano update dictionaries for training.

This function implements different methods to control the learning rate.

Update Functions:-

sgd - Stochastic Gradient Descent (SGD) updates.

momentum - Stochastic Gradient Descent (SGD) updates with momentum.

nesterov_momentum - Stochastic Gradient Descent (SGD) updates with Nesterov momentum.

adagrad - Adagrad updates.

rmsprop - RMSProp updates.

adadelta - Adadelta updates.

adam - Adam updates.

adamax - Adamax updates.

amsgrad - AMSGrad updates.

Links for the more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/neural-network-projects

Update Functions Example:-

import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.updates import nesterov_momentum
ly_in = InputLayer((100, 20))
ly1 = DenseLayer(ly_in, num_units=3, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(ly1, x)
params = lasagne.layers.get_all_params(ly1)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
updates = nesterov_momentum(loss, params, learning_rate=1e-4, momentum=.9)
train_fn = theano.function([x, y], updates=updates)

updates = lasagne.updates.rmsprop(loss, params, learning_rate=0.0001)
updates = lasagne.updates.apply_momentum(updates, params, momentum=0.9)
updates

OrderedDict([(<TensorType(float64, matrix)>, Elemwise{add,no_inplace}.0),
             (W, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{add,no_inplace}.0),
             (b, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, matrix)>, Elemwise{sub,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

What Users are saying..

profile image

Ray han

Tech Leader | Stanford / Yale University
linkedin profile url

I think that they are fantastic. I attended Yale and Stanford and have worked at Honeywell,Oracle, and Arthur Andersen(Accenture) in the US. I have taken Big Data and Hadoop,NoSQL, Spark, Hadoop... Read More

Relevant Projects

Learn How to Build PyTorch Neural Networks from Scratch
In this deep learning project, you will learn how to build PyTorch neural networks from scratch.

Recommender System Machine Learning Project for Beginners-4
Collaborative Filtering Recommender System Project - Comparison of different model based and memory based methods to build recommendation system using collaborative filtering.

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .

Build an AI Chatbot from Scratch using Keras Sequential Model
In this NLP Project, you will learn how to build an AI Chatbot from Scratch using Keras Sequential Model.

End-to-End ML Model Monitoring using Airflow and Docker
In this MLOps Project, you will learn to build an end to end pipeline to monitor any changes in the predictive power of model or degradation of data.

Build a Speech-Text Transcriptor with Nvidia Quartznet Model
In this Deep Learning Project, you will leverage transfer learning from Nvidia QuartzNet pre-trained models to develop a speech-to-text transcriptor.

Build CNN for Image Colorization using Deep Transfer Learning
Image Processing Project -Train a model for colorization to make grayscale images colorful using convolutional autoencoders.

Build a Logistic Regression Model in Python from Scratch
Regression project to implement logistic regression in python from scratch on streaming app data.

Digit Recognition using CNN for MNIST Dataset in Python
In this deep learning project, you will build a convolutional neural network using MNIST dataset for handwritten digit recognition.

Build a Credit Default Risk Prediction Model with LightGBM
In this Machine Learning Project, you will build a classification model for default prediction with LightGBM.