How to use LeakyRelu activation in TF learn

This recipe helps you use LeakyRelu activation in TF learn

Recipe Objective

This recipe explains how to use LeakyReLU.

Importing Libraries

We'll import tflearn, tensorflow as tf and tflearn.datasets.mnist as mnist.

import tflearn
import tensorflow as tf
import tflearn.datasets.mnist as mnist
from __future__ import division, print_function, absolute_import

LeakyReLU

LeakyReLU introduce non zero gradient for negative input. It is a modified version of ReLU.
Its syntax is: tflearn.activations.leaky_relu (x, alpha=0.1, name='LeakyReLU')
We have combined TFLearn built-in ops with Tensorflow graph. We have built this using MNIST Dataset.
To create a multilayer perceptron we have used TFLearn LeakyReLU activations ops.

with tf.Graph().as_default():

    x = tf.placeholder("float", [None, 784])
    y = tf.placeholder("float", [None, 10])
    u = tf.Variable(tf.random_normal([784, 256]))
    v = tf.Variable(tf.random_normal([256, 256]))
    w = tf.Variable(tf.random_normal([256, 10]))
    a = tf.Variable(tf.random_normal([256]))
    b = tf.Variable(tf.random_normal([256]))
    c = tf.Variable(tf.random_normal([10]))

    def net(X):
       X = tflearn.leaky_relu (tf.add(tf.matmul(X, u), a))
       tflearn.summaries.monitor_activation(x)
       X = tflearn.leaky_relu (tf.add(tf.matmul(X, v), b))
       tflearn.summaries.monitor_activation(x)
       X = tf.nn.leaky_relu (tf.add(tf.matmul(X, w), c))
       return X

    my_net = net(x)

What Users are saying..

profile image

Ray han

Tech Leader | Stanford / Yale University
linkedin profile url

I think that they are fantastic. I attended Yale and Stanford and have worked at Honeywell,Oracle, and Arthur Andersen(Accenture) in the US. I have taken Big Data and Hadoop,NoSQL, Spark, Hadoop... Read More

Relevant Projects

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

Isolation Forest Model and LOF for Anomaly Detection in Python
Credit Card Fraud Detection Project - Build an Isolation Forest Model and Local Outlier Factor (LOF) in Python to identify fraudulent credit card transactions.

LLM Project to Build and Fine Tune a Large Language Model
In this LLM project for beginners, you will learn to build a knowledge-grounded chatbot using LLM's and learn how to fine tune it.

Build a Multi-Class Classification Model in Python on Saturn Cloud
In this machine learning classification project, you will build a multi-class classification model in Python on Saturn Cloud to predict the license status of a business.

Machine Learning Project to Forecast Rossmann Store Sales
In this machine learning project you will work on creating a robust prediction model of Rossmann's daily sales using store, promotion, and competitor data.

Build a Logistic Regression Model in Python from Scratch
Regression project to implement logistic regression in python from scratch on streaming app data.

Build Portfolio Optimization Machine Learning Models in R
Machine Learning Project for Financial Risk Modelling and Portfolio Optimization with R- Build a machine learning model in R to develop a strategy for building a portfolio for maximized returns.

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .

Learn to Build a Siamese Neural Network for Image Similarity
In this Deep Learning Project, you will learn how to build a siamese neural network with Keras and Tensorflow for Image Similarity.

Learn to Build a Neural network from Scratch using NumPy
In this deep learning project, you will learn to build a neural network from scratch using NumPy