What is the use of activation functions in keras ?
MACHINE LEARNING RECIPES DATA CLEANING PYTHON DATA MUNGING PANDAS CHEATSHEET     ALL TAGS

What is the use of activation functions in keras ?

What is the use of activation functions in keras ?

This recipe explains what is the use of activation functions in keras

Recipe Objective

Activation Functions in Keras

An activation function is a mathematical **gate** in between the input feeding the current neuron and its output going to the next layer. It can be as simple as a step function that turns the neuron output on and off, depending on a rule or threshold. Or it can be a transformation that maps the input signals into output signals that are needed for the neural network to function.

3 Types of Activation Functions 1. Binary Step Function 2. Linear Activation Function 3. Non-Linear Activation Functions

Activation Functions futher divided into sub parts that we are familiar with. 1. Sigmoid / Logistic 2. TanH / Hyperbolic Tangent 3. ReLU (Rectified Linear Unit) 4. Leaky ReLU 5. Parametric ReLU 6. Softmax 7. Swish 8. Softplus

Step 1- Importing Libraries

import tensorflow as tf from tensorflow.keras import layers from tensorflow.keras import activations from keras.models import Sequential from keras.layers import Dense from keras.layers import Dropout from tensorflow.keras import layers

Step 2- Defining the model.

Defining the model and then define the layers, kernel initializer, and its input nodes shape.

#Model model = Sequential() model.add(layers.Dense(64, kernel_initializer='uniform', input_shape=(10,)))

Step 3- Defining Activation function.

We will show you how to use activation functions on some models to works

a = tf.constant([-200, -10, 0.0, 10, 200], dtype = tf.float32) b= tf.keras.activations.relu(a).numpy() print(b)
[  0.   0.   0.  10. 200.]
a = tf.constant([-200, -10.0, 0.0, 10.0, 200], dtype = tf.float32) b = tf.keras.activations.sigmoid(a).numpy() b
array([0.000000e+00, 4.539993e-05, 5.000000e-01, 9.999546e-01,
       1.000000e+00], dtype=float32)
a = tf.constant([-200, -10.0, 0.0, 10.0, 200], dtype = tf.float32) b = tf.keras.activations.softplus(a) b.numpy()
array([0.0000000e+00, 4.5398901e-05, 6.9314718e-01, 1.0000046e+01,
       2.0000000e+02], dtype=float32)
a = tf.constant([-200.0,-10.0, 0.0,10.0,200.0], dtype = tf.float32) b = tf.keras.activations.tanh(a) b.numpy()
array([-1., -1.,  0.,  1.,  1.], dtype=float32)

We passed the same input to all the activation functions to get the different outputs. So, we can easily understand and observe the difference between all the activation functions easily.

Relevant Projects

Census Income Data Set Project - Predict Adult Census Income
Use the Adult Income dataset to predict whether income exceeds 50K yr based on census data.

Predict Employee Computer Access Needs in Python
Data Science Project in Python- Given his or her job role, predict employee access needs using amazon employee database.

Loan Eligibility Prediction in Python using H2O.ai
In this loan prediction project you will build predictive models in Python using H2O.ai to predict if an applicant is able to repay the loan or not.

Walmart Sales Forecasting Data Science Project
Data Science Project in R-Predict the sales for each department using historical markdown data from the Walmart dataset containing data of 45 Walmart stores.

Abstractive Text Summarization using Transformers-BART Model
Deep Learning Project to implement an Abstractive Text Summarizer using Google's Transformers-BART Model to generate news article headlines.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Forecasting Business KPI's with Tensorflow and Python
In this machine learning project, you will use the video clip of an IPL match played between CSK and RCB to forecast key performance indicators like the number of appearances of a brand logo, the frames, and the shortest and longest area percentage in the video.

Inventory Demand Forecasting using Machine Learning in R
In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

German Credit Dataset Analysis to Classify Loan Applications
In this data science project, you will work with German credit dataset using classification techniques like Decision Tree, Neural Networks etc to classify loan applications using R.

Data Science Project in Python on BigMart Sales Prediction
The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.