What are the regularization functions in lasagne?

This recipe explains what are the regularization functions in lasagne.

Recipe Objective - What are the regularization functions in lasagne?

Lasagne provides "lasagne.regularization" class to apply regularization to the weights in a network.

Learn to Build a Multi Class Image Classification Model in Python from Scratch

Regularization Functions:-

1. lasagne.regularization.apply_penalty() - Computes the total cost for applying a specified penalty to a tensor or group of tensors.

2. lasagne.regularization.regularize_layer_params() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.

3. lasagne.regularization.regularize_layer_params_weighted() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.

4. lasagne.regularization.regularize_layer_params_weighted() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers, weighted by a coefficient for each layer.

5. lasagne.regularization.l1(x) - Computes the L1 norm of a tensor.

6. lasagne.regularization.l2(x) - Computes the L2 norm of a tensor.

For more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/tensorflow-projects

Example:-

import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.regularization import regularize_layer_params_weighted, l2, l1
from lasagne.regularization import regularize_layer_params
layer_in = InputLayer((100, 20))
layer_1 = DenseLayer(layer_in, num_units=3)
layer_2 = DenseLayer(layer1, num_units=5, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(layer2, x)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
layers = {layer_1: 0.1, layer_2: 0.5}
l2_penalty = regularize_layer_params_weighted(layers, l2)
l1_penalty = regularize_layer_params(layer2, l1) * 1e-4
loss = loss + l2_penalty + l1_penalty

loss

Elemwise{add,no_inplace}.0

What Users are saying..

profile image

Jingwei Li

Graduate Research assistance at Stony Brook University
linkedin profile url

ProjectPro is an awesome platform that helps me learn much hands-on industrial experience with a step-by-step walkthrough of projects. There are two primary paths to learn: Data Science and Big Data.... Read More

Relevant Projects

Mastering A/B Testing: A Practical Guide for Production
In this A/B Testing for Machine Learning Project, you will gain hands-on experience in conducting A/B tests, analyzing statistical significance, and understanding the challenges of building a solution for A/B testing in a production environment.

NLP and Deep Learning For Fake News Classification in Python
In this project you will use Python to implement various machine learning methods( RNN, LSTM, GRU) for fake news classification.

MLOps Project for a Mask R-CNN on GCP using uWSGI Flask
MLOps on GCP - Solved end-to-end MLOps Project to deploy a Mask RCNN Model for Image Segmentation as a Web Application using uWSGI Flask, Docker, and TensorFlow.

Deep Learning Project for Time Series Forecasting in Python
Deep Learning for Time Series Forecasting in Python -A Hands-On Approach to Build Deep Learning Models (MLP, CNN, LSTM, and a Hybrid Model CNN-LSTM) on Time Series Data.

MLOps AWS Project on Topic Modeling using Gunicorn Flask
In this project we will see the end-to-end machine learning development process to design, build and manage reproducible, testable, and evolvable machine learning models by using AWS

Word2Vec and FastText Word Embedding with Gensim in Python
In this NLP Project, you will learn how to use the popular topic modelling library Gensim for implementing two state-of-the-art word embedding methods Word2Vec and FastText models.

Build a Face Recognition System in Python using FaceNet
In this deep learning project, you will build your own face recognition system in Python using OpenCV and FaceNet by extracting features from an image of a person's face.

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction

Many-to-One LSTM for Sentiment Analysis and Text Generation
In this LSTM Project , you will build develop a sentiment detection model using many-to-one LSTMs for accurate prediction of sentiment labels in airline text reviews. Additionally, we will also train many-to-one LSTMs on 'Alice's Adventures in Wonderland' to generate contextually relevant text.

Image Classification Model using Transfer Learning in PyTorch
In this PyTorch Project, you will build an image classification model in PyTorch using the ResNet pre-trained model.