How to create a custom layer in tf to apply on a dataset in tf

This recipe helps you create a custom layer in tf to apply on a dataset in tf

Recipe Objective

How to create a custom layer in tf to apply on a dataset in tf?

This can be done with the help of "keras.layers" package available in tensorflow. In this package layers are objects for creating a new layer simply construct the object. Most of the layers takes as a first argument number of output channels. The number of the input dimensions are often unnecessary as it can be complete, the first time the layer is used but it can be provided if we want to specify it manually, which can be useful in some complex models. These layers have many useful methods such as we can inspect a layer using a variable In this case a fully connected layer will have variables for weights and biases.

Step 1 - Import library

import tensorflow as tf

Step 2 - Check the GPU

print(tf.test.is_gpu_available())

WARNING:tensorflow:From :1: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
True

Step 3 - Add layers

add_layers = tf.keras.layers.Dense(100) add_layers = tf.keras.layers.Dense(10, input_shape=(None, 5))

Step 4 - Call the layer

add_layers(tf.zeros([10, 5]))

 

Step 5 - Inspect all variables

add_layers.variables

[<tf.Variable 'dense_1/kernel:0' shape=(5, 10) dtype=float32, numpy=
 array([[ 0.49328917,  0.22779489,  0.43836445, -0.57288927,  0.45334858,
          0.13166016,  0.47832364, -0.4829657 , -0.5241113 ,  0.30270725],
        [ 0.06990689, -0.11654478, -0.03789103,  0.3210559 ,  0.35780483,
          0.5890518 ,  0.23949146,  0.5354218 , -0.53284043,  0.43305403],
        [ 0.5185184 , -0.1057139 , -0.5638231 ,  0.23032427,  0.33610386,
         -0.5105903 ,  0.07079613,  0.29798406, -0.08220285, -0.6313142 ],
        [-0.6187295 ,  0.12495577, -0.05369467,  0.10245973, -0.04212612,
         -0.4085849 , -0.18551952, -0.08891177,  0.17679864,  0.42220014],
        [ 0.0275442 ,  0.5392565 , -0.31567407, -0.15756017, -0.3372639 ,
         -0.5478669 , -0.06288964, -0.1057961 , -0.32654127, -0.4649645 ]],
       dtype=float32)>,
 <tf.Variable 'dense_1/bias:0' shape=(10,) dtype=float32, numpy=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)>]

Step 6 - Check the accessors

print("check the layer kernel:",add_layers.kernel) print("Layer bias:",add_layers.bias)

check the layer kernel: <tf.Variable 'dense_1/kernel:0' shape=(5, 10) dtype=float32, numpy=
array([[ 0.49328917,  0.22779489,  0.43836445, -0.57288927,  0.45334858,
         0.13166016,  0.47832364, -0.4829657 , -0.5241113 ,  0.30270725],
       [ 0.06990689, -0.11654478, -0.03789103,  0.3210559 ,  0.35780483,
         0.5890518 ,  0.23949146,  0.5354218 , -0.53284043,  0.43305403],
       [ 0.5185184 , -0.1057139 , -0.5638231 ,  0.23032427,  0.33610386,
        -0.5105903 ,  0.07079613,  0.29798406, -0.08220285, -0.6313142 ],
       [-0.6187295 ,  0.12495577, -0.05369467,  0.10245973, -0.04212612,
        -0.4085849 , -0.18551952, -0.08891177,  0.17679864,  0.42220014],
       [ 0.0275442 ,  0.5392565 , -0.31567407, -0.15756017, -0.3372639 ,
        -0.5478669 , -0.06288964, -0.1057961 , -0.32654127, -0.4649645 ]],
      dtype=float32)>
Layer bias: <tf.Variable 'dense_1/bias:0' shape=(10,) dtype=float32, numpy=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)>

Step 7 - Implement custom layers

def call(self, input): return tf.matmul(input, self.kernel) add_layers = MyDenseLayer(10)

Here we are extending the tf.keras.layer class and implementing it, the "init" is where the input variable initialization is done, "build" where we can do the rest initialization and can have the shape of input tensors, and for the forwards computation "call" is there.

Step 8 - Call the build layer

zeros_layer = add_layers(tf.zeros([10, 5]))

Step 9 - Print Results

print([var.name for var in add_layers.trainable_variables])

['my_dense_layer/kernel:0']

{"mode":"full","isActive":false}

What Users are saying..

profile image

Ed Godalle

Director Data Analytics at EY / EY Tech
linkedin profile url

I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More

Relevant Projects

Build an optimal End-to-End MLOps Pipeline and Deploy on GCP
Learn how to build and deploy an end-to-end optimal MLOps Pipeline for Loan Eligibility Prediction Model in Python on GCP

Learn to Build an End-to-End Machine Learning Pipeline - Part 1
In this Machine Learning Project, you will learn how to build an end-to-end machine learning pipeline for predicting truck delays, addressing a major challenge in the logistics industry.

Azure Deep Learning-Deploy RNN CNN models for TimeSeries
In this Azure MLOps Project, you will learn to perform docker-based deployment of RNN and CNN Models for Time Series Forecasting on Azure Cloud.

Machine Learning Project to Forecast Rossmann Store Sales
In this machine learning project you will work on creating a robust prediction model of Rossmann's daily sales using store, promotion, and competitor data.

Classification Projects on Machine Learning for Beginners - 2
Learn to implement various ensemble techniques to predict license status for a given business.

Recommender System Machine Learning Project for Beginners-2
Recommender System Machine Learning Project for Beginners Part 2- Learn how to build a recommender system for market basket analysis using association rule mining.

CycleGAN Implementation for Image-To-Image Translation
In this GAN Deep Learning Project, you will learn how to build an image to image translation model in PyTorch with Cycle GAN.

BigMart Sales Prediction ML Project in Python
The goal of the BigMart Sales Prediction ML project is to build and evaluate different predictive models and determine the sales of each product at a store.

PyTorch Project to Build a LSTM Text Classification Model
In this PyTorch Project you will learn how to build an LSTM Text Classification model for Classifying the Reviews of an App .

Build Regression (Linear,Ridge,Lasso) Models in NumPy Python
In this machine learning regression project, you will learn to build NumPy Regression Models (Linear Regression, Ridge Regression, Lasso Regression) from Scratch.