What are the updates modification functions in lasagne?

This recipe explains what are the updates modification functions in lasagne.

Recipe Objective - What are the updates modification functions in lasagne?

Updates Modification Functions:-

1. apply_momentum - Returns a modified update dictionary including momentum.

2. apply_nesterov_momentum - Returns a modified update dictionary including Nesterov momentum.

Access YOLO OCR Character Recognition Project with Source Code

For more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/tensorflow-projects

Update Modification Functions Example:-

import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.updates import nesterov_momentum
ly_in = InputLayer((100, 20))
ly1 = DenseLayer(ly_in, num_units=3, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(ly1, x)
params = lasagne.layers.get_all_params(ly1)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
updates = nesterov_momentum(loss, params, learning_rate=1e-4, momentum=.9)
train_fn = theano.function([x, y], updates=updates)

updates = lasagne.updates.rmsprop(loss, params, learning_rate=0.0001)
updates = lasagne.updates.apply_momentum(updates, params, momentum=0.9)
updates

OrderedDict([(<TensorType(float64, matrix)>, Elemwise{add,no_inplace}.0),
             (W, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{add,no_inplace}.0),
             (b, Elemwise{add,no_inplace}.0),
             (<TensorType(float64, matrix)>, Elemwise{sub,no_inplace}.0),
             (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])

What Users are saying..

profile image

Abhinav Agarwal

Graduate Student at Northwestern University
linkedin profile url

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Locality Sensitive Hashing Python Code for Look-Alike Modelling
In this deep learning project, you will find similar images (lookalikes) using deep learning and locality sensitive hashing to find customers who are most likely to click on an ad.

Isolation Forest Model and LOF for Anomaly Detection in Python
Credit Card Fraud Detection Project - Build an Isolation Forest Model and Local Outlier Factor (LOF) in Python to identify fraudulent credit card transactions.

Topic modelling using Kmeans clustering to group customer reviews
In this Kmeans clustering machine learning project, you will perform topic modelling in order to group customer reviews based on recurring patterns.

Build a Similar Images Finder with Python, Keras, and Tensorflow
Build your own image similarity application using Python to search and find images of products that are similar to any given product. You will implement the K-Nearest Neighbor algorithm to find products with maximum similarity.

Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN

Learn How to Build PyTorch Neural Networks from Scratch
In this deep learning project, you will learn how to build PyTorch neural networks from scratch.

AWS MLOps Project for Gaussian Process Time Series Modeling
MLOps Project to Build and Deploy a Gaussian Process Time Series Model in Python on AWS

Machine Learning Project to Forecast Rossmann Store Sales
In this machine learning project you will work on creating a robust prediction model of Rossmann's daily sales using store, promotion, and competitor data.

Build an End-to-End AWS SageMaker Classification Model
MLOps on AWS SageMaker -Learn to Build an End-to-End Classification Model on SageMaker to predict a patient’s cause of death.

Learn Hyperparameter Tuning for Neural Networks with PyTorch
In this Deep Learning Project, you will learn how to optimally tune the hyperparameters (learning rate, epochs, dropout, early stopping) of a neural network model in PyTorch to improve model performance.