What is fine tuning in neural networks

This recipe explains what is fine tuning in neural networks

Recipe Objective - What is fine-tuning in neural networks?

Fine-tuning is a technique of model reusability in addition to feature extraction. Fine-tuning consists of unfreezing few of the top layers of the frozen model base in neural network used for feature extraction and jointly training both the newly added part of the model (for example, a fully connected classifier) and the top layers. This technique is called fine-tuning as it slightly adjusts the more abstract representations of fine-tuning model being reused so that it can be made more relevant for the problem at hand. Only the top layers of the convolutional base are possible to be fine-tune once the classifier on top has already been trained because to be able to train a randomly initialized classifier, freezing of pretrained convnets like VGG16 should have to done.

This recipe explains what is fine-tuning, explaining its benefits and how it can be executed.

Explore the BERT Variants - ALBERT vs DistilBERT

Explanation of Fine-tuning.

Fine-tuning is the technique used by many data scientist in the top competitions organized on Kaggle and various other platforms. Steps to fine-tune a network are as follows:- 1. Add custom network on top of an already-trained base network. 2. Freezing the base network. 3. Training the part added. 4. Unfreezing some layers in base network. 5. Jointly train both of these layers and the part added. It is useful to fine-tune more specialized features as these are the ones that need to be repurposed on the new problem. Fast-decreasing returns in fine-tuning lower layers will occur.

In fine-tuning, the more parameters are trained, the more the risk of overfitting in neural network. Considering a convolutional base of 15 million parameters like VGG16 pretrained convnet then it would be risky to attempt to train it on a small dataset.

What Users are saying..

profile image

Anand Kumpatla

Sr Data Scientist @ Doubleslash Software Solutions Pvt Ltd
linkedin profile url

ProjectPro is a unique platform and helps many people in the industry to solve real-life problems with a step-by-step walkthrough of projects. A platform with some fantastic resources to gain... Read More

Relevant Projects

Build Regression Models in Python for House Price Prediction
In this Machine Learning Regression project, you will build and evaluate various regression models in Python for house price prediction.

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Build a Graph Based Recommendation System in Python-Part 2
In this Graph Based Recommender System Project, you will build a recommender system project for eCommerce platforms and learn to use FAISS for efficient similarity search.

Build OCR from Scratch Python using YOLO and Tesseract
In this deep learning project, you will learn how to build your custom OCR (optical character recognition) from scratch by using Google Tesseract and YOLO to read the text from any images.

Build CI/CD Pipeline for Machine Learning Projects using Jenkins
In this project, you will learn how to create a CI/CD pipeline for a search engine application using Jenkins.

Build Customer Propensity to Purchase Model in Python
In this machine learning project, you will learn to build a machine learning model to estimate customer propensity to purchase.

MLOps Project for a Mask R-CNN on GCP using uWSGI Flask
MLOps on GCP - Solved end-to-end MLOps Project to deploy a Mask RCNN Model for Image Segmentation as a Web Application using uWSGI Flask, Docker, and TensorFlow.

A/B Testing Approach for Comparing Performance of ML Models
The objective of this project is to compare the performance of BERT and DistilBERT models for building an efficient Question and Answering system. Using A/B testing approach, we explore the effectiveness and efficiency of both models and determine which one is better suited for Q&A tasks.

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

MLOps Project on GCP using Kubeflow for Model Deployment
MLOps using Kubeflow on GCP - Build and deploy a deep learning model on Google Cloud Platform using Kubeflow pipelines in Python