How to use Spacy lemmatizer?
MACHINE LEARNING RECIPES DATA CLEANING PYTHON DATA MUNGING PANDAS CHEATSHEET     ALL TAGS

How to use Spacy lemmatizer?

How to use Spacy lemmatizer?

This recipe helps you use Spacy lemmatizer

0

Recipe Objective

How to use Spacy lemmatizer, As we have discussed earlier only what is Spacy and what is lemmatizer. Spacy Lemmatization which gives the lemma of the word, lemma is nothing the but base word which has been converted through the process of lemmatization for e.g 'hostorical', 'history' will become 'history' so the lemma is 'history' here.

Step 1 - Import Spacy

import spacy

Step 2 - Initialize the Spacy en model.

load_model = spacy.load('en', disable = ['parser','ner'])

In the above code we have initialized the Spacy model and kept only the things which is required for lemmatization which is nothing but the tagger and disabled the parser and ner which are not required for now.

Step 3 - Take a simple text for sample

My_text = "This is just a sample text for the purpose of testing"

Step 4 - Parse the text

doc = load_model(My_text)

here in the above we have parsed the text that we have taken for sample by using the model that we have initialized i.e load_model.

Step 5 - Extract the lemma for each token

" ".join([token.lemma_ for token in doc])

So from the above we can see that there is slight change in the sentence which is, original - 'this is' and extracted - 'this be'.

Step 6 - Lets try with another example

text2 = "I am hanging out in a garden" doc2 = load_model(text2) " ".join([token.lemma_ for token in doc2])

So we can see whenever Spacy detects a pronoun it adds a -PRON- in the text.

Relevant Projects

Machine Learning or Predictive Models in IoT - Energy Prediction Use Case
In this machine learning and IoT project, we are going to test out the experimental data using various predictive models and train the models and break the energy usage.

Predict Credit Default | Give Me Some Credit Kaggle
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Predict Employee Computer Access Needs in Python
Data Science Project in Python- Given his or her job role, predict employee access needs using amazon employee database.

Human Activity Recognition Using Smartphones Data Set
In this deep learning project, you will build a classification system where to precisely identify human fitness activities.

Credit Card Fraud Detection as a Classification Problem
In this data science project, we will predict the credit card fraud in the transactional dataset using some of the predictive models.

Natural language processing Chatbot application using NLTK for text classification
In this NLP AI application, we build the core conversational engine for a chatbot. We use the popular NLTK text classification library to achieve this.

Music Recommendation System Project using Python and R
Machine Learning Project - Work with KKBOX's Music Recommendation System dataset to build the best music recommendation engine.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Forecast Inventory demand using historical sales data in R
In this machine learning project, you will develop a machine learning model to accurately forecast inventory demand based on historical sales data.

Data Science Project on Wine Quality Prediction in R
In this R data science project, we will explore wine dataset to assess red wine quality. The objective of this data science project is to explore which chemical properties will influence the quality of red wines.