How to use Spacy lemmatizer?

How to use Spacy lemmatizer?

How to use Spacy lemmatizer?

This recipe helps you use Spacy lemmatizer


Recipe Objective

How to use Spacy lemmatizer, As we have discussed earlier only what is Spacy and what is lemmatizer. Spacy Lemmatization which gives the lemma of the word, lemma is nothing the but base word which has been converted through the process of lemmatization for e.g 'hostorical', 'history' will become 'history' so the lemma is 'history' here.

Step 1 - Import Spacy

import spacy

Step 2 - Initialize the Spacy en model.

load_model = spacy.load('en', disable = ['parser','ner'])

In the above code we have initialized the Spacy model and kept only the things which is required for lemmatization which is nothing but the tagger and disabled the parser and ner which are not required for now.

Step 3 - Take a simple text for sample

My_text = "This is just a sample text for the purpose of testing"

Step 4 - Parse the text

doc = load_model(My_text)

here in the above we have parsed the text that we have taken for sample by using the model that we have initialized i.e load_model.

Step 5 - Extract the lemma for each token

" ".join([token.lemma_ for token in doc])

So from the above we can see that there is slight change in the sentence which is, original - 'this is' and extracted - 'this be'.

Step 6 - Lets try with another example

text2 = "I am hanging out in a garden" doc2 = load_model(text2) " ".join([token.lemma_ for token in doc2])

So we can see whenever Spacy detects a pronoun it adds a -PRON- in the text.

Relevant Projects

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Data Science Project - Instacart Market Basket Analysis
Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.

Perform Time series modelling using Facebook Prophet
In this project, we are going to talk about Time Series Forecasting to predict the electricity requirement for a particular house using Prophet.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Human Activity Recognition Using Multiclass Classification in Python
In this human activity recognition project, we use multiclass classification machine learning techniques to analyse fitness dataset from a smartphone tracker.

Machine Learning or Predictive Models in IoT - Energy Prediction Use Case
In this machine learning and IoT project, we are going to test out the experimental data using various predictive models and train the models and break the energy usage.

Ecommerce product reviews - Pairwise ranking and sentiment analysis
This project analyzes a dataset containing ecommerce product reviews. The goal is to use machine learning models to perform sentiment analysis on product reviews and rank them based on relevance. Reviews play a key role in product recommendation systems.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.