What are the regularization functions in lasagne?

This recipe explains what are the regularization functions in lasagne.

Recipe Objective - What are the regularization functions in lasagne?

Lasagne provides "lasagne.regularization" class to apply regularization to the weights in a network.

Learn to Build a Multi Class Image Classification Model in Python from Scratch

Regularization Functions:-

1. lasagne.regularization.apply_penalty() - Computes the total cost for applying a specified penalty to a tensor or group of tensors.

2. lasagne.regularization.regularize_layer_params() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.

3. lasagne.regularization.regularize_layer_params_weighted() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers.

4. lasagne.regularization.regularize_layer_params_weighted() - Computes a regularization cost by applying a penalty to the parameters of a layer or group of layers, weighted by a coefficient for each layer.

5. lasagne.regularization.l1(x) - Computes the L1 norm of a tensor.

6. lasagne.regularization.l2(x) - Computes the L2 norm of a tensor.

For more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/tensorflow-projects

Example:-

import lasagne
import theano.tensor as T
import theano
from lasagne.nonlinearities import softmax
from lasagne.layers import InputLayer, DenseLayer, get_output
from lasagne.regularization import regularize_layer_params_weighted, l2, l1
from lasagne.regularization import regularize_layer_params
layer_in = InputLayer((100, 20))
layer_1 = DenseLayer(layer_in, num_units=3)
layer_2 = DenseLayer(layer1, num_units=5, nonlinearity=softmax)
x = T.matrix('x') # shp: num_batch x num_features
y = T.ivector('y') # shp: num_batch
ly_out = get_output(layer2, x)
loss = T.mean(T.nnet.categorical_crossentropy(ly_out, y))
layers = {layer_1: 0.1, layer_2: 0.5}
l2_penalty = regularize_layer_params_weighted(layers, l2)
l1_penalty = regularize_layer_params(layer2, l1) * 1e-4
loss = loss + l2_penalty + l1_penalty

loss

Elemwise{add,no_inplace}.0

What Users are saying..

profile image

Gautam Vermani

Data Consultant at Confidential
linkedin profile url

Having worked in the field of Data Science, I wanted to explore how I can implement projects in other domains, So I thought of connecting with ProjectPro. A project that helped me absorb this topic... Read More

Relevant Projects

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.

Learn Object Tracking (SOT, MOT) using OpenCV and Python
Get Started with Object Tracking using OpenCV and Python - Learn to implement Multiple Instance Learning Tracker (MIL) algorithm, Generic Object Tracking Using Regression Networks Tracker (GOTURN) algorithm, Kernelized Correlation Filters Tracker (KCF) algorithm, Tracking, Learning, Detection Tracker (TLD) algorithm for single and multiple object tracking from various video clips.

Learn to Build a Siamese Neural Network for Image Similarity
In this Deep Learning Project, you will learn how to build a siamese neural network with Keras and Tensorflow for Image Similarity.

Build a Autoregressive and Moving Average Time Series Model
In this time series project, you will learn to build Autoregressive and Moving Average Time Series Models to forecast future readings, optimize performance, and harness the power of predictive analytics for sensor data.

Model Deployment on GCP using Streamlit for Resume Parsing
Perform model deployment on GCP for resume parsing model using Streamlit App.

Build a Review Classification Model using Gated Recurrent Unit
In this Machine Learning project, you will build a classification model in python to classify the reviews of an app on a scale of 1 to 5 using Gated Recurrent Unit.

MLOps Project to Deploy Resume Parser Model on Paperspace
In this MLOps project, you will learn how to deploy a Resume Parser Streamlit Application on Paperspace Private Cloud.

Medical Image Segmentation Deep Learning Project
In this deep learning project, you will learn to implement Unet++ models for medical image segmentation to detect and classify colorectal polyps.

Deep Learning Project- Real-Time Fruit Detection using YOLOv4
In this deep learning project, you will learn to build an accurate, fast, and reliable real-time fruit detection system using the YOLOv4 object detection model for robotic harvesting platforms.

Build CNN Image Classification Models for Real Time Prediction
Image Classification Project to build a CNN model in Python that can classify images into social security cards, driving licenses, and other key identity information.