How to stem non english words in nlp

This recipe helps you stem non english words in nlp

Get access to Data Science projects View all Data Science projects
MACHINE LEARNING RECIPES DATA CLEANING PYTHON DATA MUNGING PANDAS CHEATSHEET     ALL TAGS

Recipe Objective

How to stem non english words?

Stemming as we have discussed already what is stemming which is nothing but reducing the words to their root size. We have seen stemming for English words, but what about non - english language words, there are stemmers available for non - english words as well. Lets understand this with practical implementation.

Build a Chatbot in Python from Scratch!

Table of Contents

Step 1 - Import the German language Stemmer

from nltk.stem.snowball import GermanStemmer

Step 2 - Store the german stemmer in a variable

german_st = GermanStemmer()

Step 3 - Take sample words

token_sample = ["Schreiben","geschrieben"]

Here we have taken some sample words in german whose english translation is:

Schreiben - writing

geschrieben - written

Step 4 - Apply stemming and print the results

stem_words = [german_st.stem(words) for words in token_sample] print("Print the output after stemming:",stem_words)

Print the output after stemming: ['schreib', 'geschrieb']

Here we can see the output as, 'schreib', 'geschrieb' whose english translation is:

schreib - write

geschrieb - wrote

So we can see the difference between our sample token words and results after applying stremming on that words.

What Users are saying..

profile image

Anand Kumpatla

Sr Data Scientist @ Doubleslash Software Solutions Pvt Ltd
linkedin profile url

ProjectPro is a unique platform and helps many people in the industry to solve real-life problems with a step-by-step walkthrough of projects. A platform with some fantastic resources to gain... Read More

Relevant Projects

Machine Learning Projects
Data Science Projects
Python Projects for Data Science
Data Science Projects in R
Machine Learning Projects for Beginners
Deep Learning Projects
Neural Network Projects
Tensorflow Projects
NLP Projects
Kaggle Projects
IoT Projects
Big Data Projects
Hadoop Real-Time Projects Examples
Spark Projects
Data Analytics Projects for Students

You might also like

Data Science Tutorial
Data Scientist Salary
How to Become a Data Scientist
Data Analyst vs Data Scientist
Data Scientist Resume
Data Science Projects for Beginners
Machine Learning Engineer
Machine Learning Projects for Beginners
Datasets
Pandas Dataframe
Machine Learning Algorithms
Regression Analysis
MNIST Dataset
Data Science Interview Questions
Python Data Science Interview Questions
Spark Interview Questions
Hadoop Interview Questions
Data Analyst Interview Questions
Machine Learning Interview Questions
AWS vs Azure
Hadoop Architecture
Spark Architecture

Relevant Projects

Deep Learning Project for Time Series Forecasting in Python
Deep Learning for Time Series Forecasting in Python -A Hands-On Approach to Build Deep Learning Models (MLP, CNN, LSTM, and a Hybrid Model CNN-LSTM) on Time Series Data.
View Project Details
Hands-On Approach to Master PyTorch Tensors with Examples
In this deep learning project, you will learn how to perform various operations on the building block of PyTorch : Tensors.
View Project Details
Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN
View Project Details
BERT Text Classification using DistilBERT and ALBERT Models
This Project Explains how to perform Text Classification using ALBERT and DistilBERT
View Project Details
Learn to Build a Neural network from Scratch using NumPy
In this deep learning project, you will learn to build a neural network from scratch using NumPy
View Project Details
Hands-On Approach to Causal Inference in Machine Learning
In this Machine Learning Project, you will learn to implement various causal inference techniques in Python to determine, how effective the sprinkler is in making the grass wet.
View Project Details
Create Your First Chatbot with RASA NLU Model and Python
Learn the basic aspects of chatbot development and open source conversational AI RASA to create a simple AI powered chatbot on your own.
View Project Details
Build an Image Segmentation Model using Amazon SageMaker
In this Machine Learning Project, you will learn to implement the UNet Architecture and build an Image Segmentation Model using Amazon SageMaker
View Project Details
Recommender System Machine Learning Project for Beginners-4
Collaborative Filtering Recommender System Project - Comparison of different model based and memory based methods to build recommendation system using collaborative filtering.
View Project Details
A/B Testing Approach for Comparing Performance of ML Models
The objective of this project is to compare the performance of BERT and DistilBERT models for building an efficient Question and Answering system. Using A/B testing approach, we explore the effectiveness and efficiency of both models and determine which one is better suited for Q&A tasks.
View Project Details