How does the GrabCut algorithm work in OpenCV

This recipe explains how does the GrabCut algorithm work in OpenCV

Recipe Objective: How does the GrabCut algorithm work in OpenCV?

This recipe explains in detail about GrabCut algorithm and how to implement it using OpenCV.

Scaling Data with FEAST Feature Store for Machine Learning

Step 1: Import the libraries and read the image.

Let us first import the necessary libraries and read the image. The image that we are using here is the one shown below.

Input Image

import cv2
import numpy as np
from matplotlib import pyplot as plt
image = cv2.imread("edgeflower.jpg")

Step 2: Understanding GrabCut algorithm

GrabCut algorithm is essentially used for interactive foreground extraction from any given image. This algorithm was structured by Carsten Rother, Vladimir Kolmogorov & Andrew Blake from Microsoft Research Cambridge.

Let us first understand how the algorithm works from a user's perspective. There are two ways in which this algorithm can work.

One way is by specifying a rectangular area in the image. The user initially feeds the algorithm with an image and the coordinates of a rectangular area in the image, which captures the foreground object completely. The area outside the rectangle is taken as a sure background. Relative to this background, the algorithm carves out the foreground object from the area within the rectangle. This process is analogous to clustering in machine learning.

The other way is by using an image mask. The user can manually create a mask for the image using any photo editing tool such as Microsoft Paint or Adobe photoshop. This mask should contain the foreground object area in white, the unwanted background area in black. This mask can be fed to the algorithm and the image the algorithm sketches out the foreground object relative to the mask.

The GrabCut Algorithm uses Gaussian Mixture Model(GMM) model behind the hood to learn and carve the foreground object.

Step 3: GrabCut algorithm implementation

In this example, let us extract the foreground using the first approach discussed in the previous step.

The foreground can be extracted using the cv2.grabCut() function, which takes the following arguments

  • img: The input image
  • mask: The mask of the image
  • bgdModel, fgdModel: These are arrays that the algorithm uses internally. We can pass an array of zeros in shape [1,65]
  • iterCount: Specifies the number of iterations the algorithm should run. The ideal value is five, but we can modify it as we want.
  • mode: The type of approach to be followed. There are two options for this parameter. They are cv.GC_INIT_WITH_RECT or cv.GC_INIT_WITH_MASK

The function returns the altered mask of the image with four classes. They are :

  • cv.GC_BGD: Sure background
  • cv.GC_FGD: Sure foreground
  • cv.GC_PR_BGD: Probable background
  • cv.GC_PR_FGD: Probable foreground

These classes can also be represented as 0,1,2,3

The function also returns the altered bgdModel and fgdModel arrays.

Let us begin the implementation by initializing the mask, fgdModel, and bgdModel variables using the np.zeros() function in NumPy.

mask = np.zeros(image.shape[:2],np.uint8)
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

Let us also enter the coordinates of the rectangular area which encloses the foreground of the image. The coordinates are in the order of (x, y, width, height)

rect = (100,70,650,600)

Now let us pass all the initialized variables to the cv2.grabCut() function along with the image.

new_mask, fg, bg = cv2.grabCut(image,mask,rect,bgdModel,fgdModel,10,cv2.GC_INIT_WITH_RECT)

Now let us combine the obtained mask with our original image

mask2 = np.where((new_mask==2)|(new_mask==0),0,1).astype('uint8')
new_image = image*mask2[:,:,np.newaxis]

Step 4: Displaying the output

Let us display the output image using matplotlib.

plt.figure(figsize=(13,5))
plt.imshow(cv2.cvtColor(new_image,cv2.COLOR_BGR2RGB))
plt.xticks([])
plt.yticks([])
plt.tight_layout()
plt.show()

Output:

=Output

As we can see, the foreground object has been carved out perfectly.

Download Materials

What Users are saying..

profile image

Jingwei Li

Graduate Research assistance at Stony Brook University
linkedin profile url

ProjectPro is an awesome platform that helps me learn much hands-on industrial experience with a step-by-step walkthrough of projects. There are two primary paths to learn: Data Science and Big Data.... Read More

Relevant Projects

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction

Deep Learning Project for Text Detection in Images using Python
CV2 Text Detection Code for Images using Python -Build a CRNN deep learning model to predict the single-line text in a given image.

Build a Logistic Regression Model in Python from Scratch
Regression project to implement logistic regression in python from scratch on streaming app data.

Personalized Medicine: Redefining Cancer Treatment
In this Personalized Medicine Machine Learning Project you will learn to classify genetic mutations on the basis of medical literature into 9 classes.

Build an AI Chatbot from Scratch using Keras Sequential Model
In this NLP Project, you will learn how to build an AI Chatbot from Scratch using Keras Sequential Model.

Create Your First Chatbot with RASA NLU Model and Python
Learn the basic aspects of chatbot development and open source conversational AI RASA to create a simple AI powered chatbot on your own.

Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN

Time Series Forecasting with LSTM Neural Network Python
Deep Learning Project- Learn to apply deep learning paradigm to forecast univariate time series data.

Build CI/CD Pipeline for Machine Learning Projects using Jenkins
In this project, you will learn how to create a CI/CD pipeline for a search engine application using Jenkins.

End-to-End Snowflake Healthcare Analytics Project on AWS-1
In this Snowflake Healthcare Analytics Project, you will leverage Snowflake on AWS to predict patient length of stay (LOS) in hospitals. The prediction of LOS can help in efficient resource allocation, lower the risk of staff/visitor infections, and improve overall hospital functioning.