How does the GrabCut algorithm work in OpenCV

This recipe explains how does the GrabCut algorithm work in OpenCV

Recipe Objective: How does the GrabCut algorithm work in OpenCV?

This recipe explains in detail about GrabCut algorithm and how to implement it using OpenCV.

Scaling Data with FEAST Feature Store for Machine Learning

Step 1: Import the libraries and read the image.

Let us first import the necessary libraries and read the image. The image that we are using here is the one shown below.

Input Image

import cv2
import numpy as np
from matplotlib import pyplot as plt
image = cv2.imread("edgeflower.jpg")

Step 2: Understanding GrabCut algorithm

GrabCut algorithm is essentially used for interactive foreground extraction from any given image. This algorithm was structured by Carsten Rother, Vladimir Kolmogorov & Andrew Blake from Microsoft Research Cambridge.

Let us first understand how the algorithm works from a user's perspective. There are two ways in which this algorithm can work.

One way is by specifying a rectangular area in the image. The user initially feeds the algorithm with an image and the coordinates of a rectangular area in the image, which captures the foreground object completely. The area outside the rectangle is taken as a sure background. Relative to this background, the algorithm carves out the foreground object from the area within the rectangle. This process is analogous to clustering in machine learning.

The other way is by using an image mask. The user can manually create a mask for the image using any photo editing tool such as Microsoft Paint or Adobe photoshop. This mask should contain the foreground object area in white, the unwanted background area in black. This mask can be fed to the algorithm and the image the algorithm sketches out the foreground object relative to the mask.

The GrabCut Algorithm uses Gaussian Mixture Model(GMM) model behind the hood to learn and carve the foreground object.

Step 3: GrabCut algorithm implementation

In this example, let us extract the foreground using the first approach discussed in the previous step.

The foreground can be extracted using the cv2.grabCut() function, which takes the following arguments

  • img: The input image
  • mask: The mask of the image
  • bgdModel, fgdModel: These are arrays that the algorithm uses internally. We can pass an array of zeros in shape [1,65]
  • iterCount: Specifies the number of iterations the algorithm should run. The ideal value is five, but we can modify it as we want.
  • mode: The type of approach to be followed. There are two options for this parameter. They are cv.GC_INIT_WITH_RECT or cv.GC_INIT_WITH_MASK

The function returns the altered mask of the image with four classes. They are :

  • cv.GC_BGD: Sure background
  • cv.GC_FGD: Sure foreground
  • cv.GC_PR_BGD: Probable background
  • cv.GC_PR_FGD: Probable foreground

These classes can also be represented as 0,1,2,3

The function also returns the altered bgdModel and fgdModel arrays.

Let us begin the implementation by initializing the mask, fgdModel, and bgdModel variables using the np.zeros() function in NumPy.

mask = np.zeros(image.shape[:2],np.uint8)
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

Let us also enter the coordinates of the rectangular area which encloses the foreground of the image. The coordinates are in the order of (x, y, width, height)

rect = (100,70,650,600)

Now let us pass all the initialized variables to the cv2.grabCut() function along with the image.

new_mask, fg, bg = cv2.grabCut(image,mask,rect,bgdModel,fgdModel,10,cv2.GC_INIT_WITH_RECT)

Now let us combine the obtained mask with our original image

mask2 = np.where((new_mask==2)|(new_mask==0),0,1).astype('uint8')
new_image = image*mask2[:,:,np.newaxis]

Step 4: Displaying the output

Let us display the output image using matplotlib.

plt.figure(figsize=(13,5))
plt.imshow(cv2.cvtColor(new_image,cv2.COLOR_BGR2RGB))
plt.xticks([])
plt.yticks([])
plt.tight_layout()
plt.show()

Output:

=Output

As we can see, the foreground object has been carved out perfectly.

Download Materials

What Users are saying..

profile image

Jingwei Li

Graduate Research assistance at Stony Brook University
linkedin profile url

ProjectPro is an awesome platform that helps me learn much hands-on industrial experience with a step-by-step walkthrough of projects. There are two primary paths to learn: Data Science and Big Data.... Read More

Relevant Projects

NLP and Deep Learning For Fake News Classification in Python
In this project you will use Python to implement various machine learning methods( RNN, LSTM, GRU) for fake news classification.

Build a Customer Churn Prediction Model using Decision Trees
Develop a customer churn prediction model using decision tree machine learning algorithms and data science on streaming service data.

Build a Graph Based Recommendation System in Python -Part 1
Python Recommender Systems Project - Learn to build a graph based recommendation system in eCommerce to recommend products.

Build a Multi Touch Attribution Machine Learning Model in Python
Identifying the ROI on marketing campaigns is an essential KPI for any business. In this ML project, you will learn to build a Multi Touch Attribution Model in Python to identify the ROI of various marketing efforts and their impact on conversions or sales..

Build CNN for Image Colorization using Deep Transfer Learning
Image Processing Project -Train a model for colorization to make grayscale images colorful using convolutional autoencoders.

NLP Project to Build a Resume Parser in Python using Spacy
Use the popular Spacy NLP python library for OCR and text classification to build a Resume Parser in Python.

Detectron2 Object Detection and Segmentation Example Python
Object Detection using Detectron2 - Build a Dectectron2 model to detect the zones and inhibitions in antibiogram images.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

ML Model Deployment on AWS for Customer Churn Prediction
MLOps Project-Deploy Machine Learning Model to Production Python on AWS for Customer Churn Prediction

Many-to-One LSTM for Sentiment Analysis and Text Generation
In this LSTM Project , you will build develop a sentiment detection model using many-to-one LSTMs for accurate prediction of sentiment labels in airline text reviews. Additionally, we will also train many-to-one LSTMs on 'Alice's Adventures in Wonderland' to generate contextually relevant text.