How to use Spacy lemmatizer?

This recipe helps you use Spacy lemmatizer

Recipe Objective

How to use Spacy lemmatizer, As we have discussed earlier only what is Spacy and what is lemmatizer. Spacy Lemmatization which gives the lemma of the word, lemma is nothing the but base word which has been converted through the process of lemmatization for e.g 'hostorical', 'history' will become 'history' so the lemma is 'history' here.

Learn How to Build a Simple Chatbot from Scratch in Python (using NLTK)

Step 1 - Import Spacy

import spacy

Step 2 - Initialize the Spacy en model.

load_model = spacy.load('en', disable = ['parser','ner'])

In the above code we have initialized the Spacy model and kept only the things which is required for lemmatization which is nothing but the tagger and disabled the parser and ner which are not required for now.

Step 3 - Take a simple text for sample

My_text = "This is just a sample text for the purpose of testing"

Step 4 - Parse the text

doc = load_model(My_text)

here in the above we have parsed the text that we have taken for sample by using the model that we have initialized i.e load_model.

Step 5 - Extract the lemma for each token

" ".join([token.lemma_ for token in doc])

So from the above we can see that there is slight change in the sentence which is, original - 'this is' and extracted - 'this be'.

Step 6 - Lets try with another example

text2 = "I am hanging out in a garden" doc2 = load_model(text2) " ".join([token.lemma_ for token in doc2])

So we can see whenever Spacy detects a pronoun it adds a -PRON- in the text.

What Users are saying..

profile image

Ed Godalle

Director Data Analytics at EY / EY Tech
linkedin profile url

I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More

Relevant Projects

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

Deep Learning Project for Text Detection in Images using Python
CV2 Text Detection Code for Images using Python -Build a CRNN deep learning model to predict the single-line text in a given image.

BERT Text Classification using DistilBERT and ALBERT Models
This Project Explains how to perform Text Classification using ALBERT and DistilBERT

Forecasting Business KPI's with Tensorflow and Python
In this machine learning project, you will use the video clip of an IPL match played between CSK and RCB to forecast key performance indicators like the number of appearances of a brand logo, the frames, and the shortest and longest area percentage in the video.

NLP Project on LDA Topic Modelling Python using RACE Dataset
Use the RACE dataset to extract a dominant topic from each document and perform LDA topic modeling in python.

Build an optimal End-to-End MLOps Pipeline and Deploy on GCP
Learn how to build and deploy an end-to-end optimal MLOps Pipeline for Loan Eligibility Prediction Model in Python on GCP

Langchain Project for Customer Support App in Python
In this LLM Project, you will learn how to enhance customer support interactions through Large Language Models (LLMs), enabling intelligent, context-aware responses. This Langchain project aims to seamlessly integrate LLM technology with databases, PDF knowledge bases, and audio processing agents to create a comprehensive customer support application.

Personalized Medicine: Redefining Cancer Treatment
In this Personalized Medicine Machine Learning Project you will learn to classify genetic mutations on the basis of medical literature into 9 classes.

Word2Vec and FastText Word Embedding with Gensim in Python
In this NLP Project, you will learn how to use the popular topic modelling library Gensim for implementing two state-of-the-art word embedding methods Word2Vec and FastText models.

A/B Testing Approach for Comparing Performance of ML Models
The objective of this project is to compare the performance of BERT and DistilBERT models for building an efficient Question and Answering system. Using A/B testing approach, we explore the effectiveness and efficiency of both models and determine which one is better suited for Q&A tasks.