How to add regularization to regression in keras?

Recipe Objective

Adding regularization in keras

Regularization generally reduces the overfitting of a model, it helps the model to generalize. It penalizes the model for having more weightage. There are two types of regularization parameters:- * L1 (Lasso) * L2 (Ridge) We will consider L1 for our example.

Learn About the Application of ARCH and GARCH models in Real-World

Step-1 Importing Libraries.

from sklearn.datasets import make_circles from keras.layers import Dense from keras.models import Sequential from keras.regularizers import l1 from keras.layers import Activation

Step 2- Prepare the Dataset.

# generate 2d classification dataset X, y = make_circles(n_samples=50) # split into train and test train = 15 X_train, X_test = X[:train, :], X[train:, :] y_train, y_test = y[:train], y[train:]

Step 3- Creating a Neural Network model.

We will create the Neural Network model and add the Regularizer in the input layer.

# define model model = Sequential() model.add(Dense(312, input_dim=2, activation='linear', activity_regularizer=l1(0.01))) model.add(Activation('relu')) model.add(Dense(2, activation='relu')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['mse']) model.summary()

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_4 (Dense)              (None, 312)               936       
_________________________________________________________________
activation_2 (Activation)    (None, 312)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 2)                 626       
=================================================================
Total params: 1,562
Trainable params: 1,562
Non-trainable params: 0