Relu activation function in keras and why is it used The Rectified Linear Unit is the most commonly used activation function in deep learning models. The function returns 0 if it receives any negative input, but for any positive value x it returns that value back. So it can be written as y =max(0,x) Some features of Relu function It is very easy to understand, there is no complicated maths formula behind it. It doesn't have the dying slope problem that mainly occurs in other activation functions like sigmoid or tanh. It has some variants in itself for some complicated maths like Leaky Relu and Parametric Relu.
import tensorflow as tf from tensorflow.keras import layers from tensorflow.keras import activations from keras.models import Sequential from keras.layers import Dense from keras.layers import Dropout from tensorflow.keras import layers
We will Define the model and then define the layers, kernel initializer, and its input nodes shape.
#Model model = Sequential() model.add(layers.Dense(64, kernel_initializer='uniform', input_shape=(10,)))
We will show you how to use ReLU activation functions on some models to works.
a = tf.constant([-200, -10, 0.0, 10, 200], dtype = tf.float32) b= tf.keras.activations.relu(a).numpy() print(b)
[ 0. 0. 0. 10. 200.]