What is the use of activation functions in keras ?

This recipe explains what is the use of activation functions in keras

Recipe Objective

Activation Functions in Keras

An activation function is a mathematical **gate** in between the input feeding the current neuron and its output going to the next layer. It can be as simple as a step function that turns the neuron output on and off, depending on a rule or threshold. Or it can be a transformation that maps the input signals into output signals that are needed for the neural network to function.

3 Types of Activation Functions 1. Binary Step Function 2. Linear Activation Function 3. Non-Linear Activation Functions

Activation Functions futher divided into sub parts that we are familiar with. 1. Sigmoid / Logistic 2. TanH / Hyperbolic Tangent 3. ReLU (Rectified Linear Unit) 4. Leaky ReLU 5. Parametric ReLU 6. Softmax 7. Swish 8. Softplus

Complete Guide to Tensorflow for Deep Learning with Python for Free

Step 1- Importing Libraries

import tensorflow as tf from tensorflow.keras import layers from tensorflow.keras import activations from keras.models import Sequential from keras.layers import Dense from keras.layers import Dropout from tensorflow.keras import layers

Step 2- Defining the model.

Defining the model and then define the layers, kernel initializer, and its input nodes shape.

#Model model = Sequential() model.add(layers.Dense(64, kernel_initializer='uniform', input_shape=(10,)))

Step 3- Defining Activation function.

We will show you how to use activation functions on some models to works

a = tf.constant([-200, -10, 0.0, 10, 200], dtype = tf.float32) b= tf.keras.activations.relu(a).numpy() print(b)

[  0.   0.   0.  10. 200.]

a = tf.constant([-200, -10.0, 0.0, 10.0, 200], dtype = tf.float32) b = tf.keras.activations.sigmoid(a).numpy() b

array([0.000000e+00, 4.539993e-05, 5.000000e-01, 9.999546e-01,
       1.000000e+00], dtype=float32)

a = tf.constant([-200, -10.0, 0.0, 10.0, 200], dtype = tf.float32) b = tf.keras.activations.softplus(a) b.numpy()

array([0.0000000e+00, 4.5398901e-05, 6.9314718e-01, 1.0000046e+01,
       2.0000000e+02], dtype=float32)

a = tf.constant([-200.0,-10.0, 0.0,10.0,200.0], dtype = tf.float32) b = tf.keras.activations.tanh(a) b.numpy()

array([-1., -1.,  0.,  1.,  1.], dtype=float32)

We passed the same input to all the activation functions to get the different outputs. So, we can easily understand and observe the difference between all the activation functions easily.

What Users are saying..

profile image

Jingwei Li

Graduate Research assistance at Stony Brook University
linkedin profile url

ProjectPro is an awesome platform that helps me learn much hands-on industrial experience with a step-by-step walkthrough of projects. There are two primary paths to learn: Data Science and Big Data.... Read More

Relevant Projects

Learn How to Build a Linear Regression Model in PyTorch
In this Machine Learning Project, you will learn how to build a simple linear regression model in PyTorch to predict the number of days subscribed.

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.

Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN

Loan Default Prediction Project using Explainable AI ML Models
Loan Default Prediction Project that employs sophisticated machine learning models, such as XGBoost and Random Forest and delves deep into the realm of Explainable AI, ensuring every prediction is transparent and understandable.

Loan Eligibility Prediction in Python using H2O.ai
In this loan prediction project you will build predictive models in Python using H2O.ai to predict if an applicant is able to repay the loan or not.

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction

PyTorch Project to Build a LSTM Text Classification Model
In this PyTorch Project you will learn how to build an LSTM Text Classification model for Classifying the Reviews of an App .

Learn to Build an End-to-End Machine Learning Pipeline - Part 2
In this Machine Learning Project, you will learn how to build an end-to-end machine learning pipeline for predicting truck delays, incorporating Hopsworks' feature store and Weights and Biases for model experimentation.

CycleGAN Implementation for Image-To-Image Translation
In this GAN Deep Learning Project, you will learn how to build an image to image translation model in PyTorch with Cycle GAN.

Stock Price Prediction Project using LSTM and RNN
Learn how to predict stock prices using RNN and LSTM models. Understand deep learning concepts and apply them to real-world financial data for accurate forecasting.