What are the nonlinearities functions in lasagne?

This recipe explains what are the nonlinearities functions in lasagne.

Recipe Objective - What are the nonlinearities functions in lasagne?

Lasagne provides the "lasagne.nonlinearities" class for nonlinearities function.

These functions are used for artificial neurons.

Nonlinearities Functions:-

1. sigmoid(x) - Sigmoid activation function φ(x)=1/1+e−x

2. softmax(x) - Softmax activation function φ(x)j=e^xj/∑Kk=1exk where K is the total number of neurons in the layer.

3. tanh(x) - Tanh activation function φ(x)=tanh(x)

4. ScaledTanH([scale_in, scale_out]) - Scaled tanh φ(x)=tanh(α⋅x)⋅β

5. rectify(x) - Rectify activation function φ(x)=max(0,x)

6. LeakyRectify([leakiness]) - Leaky rectifier φ(x)=(x>0)?x:α⋅x

7. leaky_rectify(x) - Instance of LeakyRectify with leakiness α=0.01

8. very_leaky_rectify(x) - Instance of LeakyRectify with leakiness α=1/3

9. elu(x) - Exponential Linear Unit φ(x)=(x>0)?x:ex−1

10. SELU([scale, scale_neg]) - Scaled Exponential Linear Unit

11. selu(x) - Instance of SELU with :math:`alphaapprox 1.6733,

12. softplus(x) - Softplus activation function φ(x)=log(1+ex)

13. linear(x) - Linear activation function φ(x)=x

14. identity(x) - Linear activation function φ(x)=x

Links for the more related projects:-

/projects/data-science-projects/deep-learning-projects
/projects/data-science-projects/neural-network-projects

from lasagne.layers import InputLayer, DenseLayer
ly_in = InputLayer((None, 100))
from lasagne.nonlinearities import ScaledTanH
scaled_tanh = ScaledTanH(scale_in=0.5, scale_out=2.27)
ly1 = DenseLayer(ly_in, num_units=200, nonlinearity=scaled_tanh)
ly1

from lasagne.nonlinearities import LeakyRectify
custom_rectify = LeakyRectify(0.1)
ly2 = DenseLayer(ly_in, num_units=200, nonlinearity=custom_rectify)
ly2

from lasagne.nonlinearities import leaky_rectify
ly3 = DenseLayer(ly_in, num_units=200, nonlinearity=leaky_rectify)
ly3

from lasagne.nonlinearities import very_leaky_rectify
ly4 = DenseLayer(ly_in, num_units=200, nonlinearity=very_leaky_rectify)
ly4

What Users are saying..

profile image

Ameeruddin Mohammed

ETL (Abintio) developer at IBM
linkedin profile url

I come from a background in Marketing and Analytics and when I developed an interest in Machine Learning algorithms, I did multiple in-class courses from reputed institutions though I got good... Read More

Relevant Projects

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

Credit Card Default Prediction using Machine learning techniques
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Build a Multi ClassText Classification Model using Naive Bayes
Implement the Naive Bayes Algorithm to build a multi class text classification model in Python.

CycleGAN Implementation for Image-To-Image Translation
In this GAN Deep Learning Project, you will learn how to build an image to image translation model in PyTorch with Cycle GAN.

Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN

Hands-On Approach to Regression Discontinuity Design Python
In this machine learning project, you will learn to implement Regression Discontinuity Design Example in Python to determine the effect of age on Mortality Rate in Python.

Build a Credit Default Risk Prediction Model with LightGBM
In this Machine Learning Project, you will build a classification model for default prediction with LightGBM.

Learn Object Tracking (SOT, MOT) using OpenCV and Python
Get Started with Object Tracking using OpenCV and Python - Learn to implement Multiple Instance Learning Tracker (MIL) algorithm, Generic Object Tracking Using Regression Networks Tracker (GOTURN) algorithm, Kernelized Correlation Filters Tracker (KCF) algorithm, Tracking, Learning, Detection Tracker (TLD) algorithm for single and multiple object tracking from various video clips.

Build Real Estate Price Prediction Model with NLP and FastAPI
In this Real Estate Price Prediction Project, you will learn to build a real estate price prediction machine learning model and deploy it on Heroku using FastAPI Framework.

Loan Eligibility Prediction Project using Machine learning on GCP
Loan Eligibility Prediction Project - Use SQL and Python to build a predictive model on GCP to determine whether an application requesting loan is eligible or not.