How to find ngrams from text using nltk

Recipe Objective

How to find n-grams from text?. first of all lets understand what is Ngram so it means the sequence of N words, for e.g "A mango" is a 2-gram, "the cat is dancing" is 4-gram and many more.

Build a Chatbot in Python from Scratch!

Step 1 - Import the necessary packages

import nltk from nltk.util import ngrams

Step 2 - Define a function for ngrams

def extract_ngrams(data, num): n_grams = ngrams(nltk.word_tokenize(data), num) return [ ' '.join(grams) for grams in n_grams]

Here we have defined a function called extract_ngrams which will generate ngrams from sentences.

Step 3 - Take a sample text

My_text = 'Jack is very good in mathematics but he is not that much good in science'

Step 4 - Print the ngrams of the sentence

print("1-gram of the sample text: ", extract_ngrams(My_text, 1), '\n') print("2-gram of the sample text: ", extract_ngrams(My_text, 2), '\n') print("3-gram of the sample text: ", extract_ngrams(My_text, 3), '\n') print("4-gram of the sample text: ", extract_ngrams(My_text, 4), '\n')

1-gram of the sample text:  ['Jack', 'is', 'very', 'good', 'in', 'mathematics', 'but', 'he', 'is', 'not', 'that', 'much', 'good', 'in', 'science'] 

2-gram of the sample text:  ['Jack is', 'is very', 'very good', 'good in', 'in mathematics', 'mathematics but', 'but he', 'he is', 'is not', 'not that', 'that much', 'much good', 'good in', 'in science'] 

3-gram of the sample text:  ['Jack is very', 'is very good', 'very good in', 'good in mathematics', 'in mathematics but', 'mathematics but he', 'but he is', 'he is not', 'is not that', 'not that much', 'that much good', 'much good in', 'good in science'] 

4-gram of the sample text:  ['Jack is very good', 'is very good in', 'very good in mathematics', 'good in mathematics but', 'in mathematics but he', 'mathematics but he is', 'but he is not', 'he is not that', 'is not that much', 'not that much good', 'that much good in', 'much good in science']

The above code will give the output for the number of ngrams that we have specified their, It can be increased or decreased as per our requirement.