Explain what is a hashing vectorizer in nlp

This recipe explains what is a hashing vectorizer in nlp

Recipe Objective

what is a hashing vectorizer?

hashing vectorizer is a vectorizer which uses the hashing trick to find the token string name to feature integer index mapping. Conversion of text documents into matrix is done by this vectorizer where it turns the collection of documents into a sparse matrix which are holding the token occurence counts. Advantages for hashing vectorizer are:

As there is no need of storing the vocabulary dictionary in the memory, for large data sets it is very low memory scalable. As there in no state during the fit, it can be used in a streaming or parallel pipeline. And more.

Learn How to use XLNet for Text Classification

Step 1 - Import the necessary libraries

from sklearn.feature_extraction.text import HashingVectorizer

Step 2 - Take a Sample text

Sample_text = ["Jon is playing football.","He loves to play football.","He is just 10 years old.", "His favorite player is Cristiano Ronaldo."] print(Sample_text)

['Jon is playing football.', 'He loves to play football.', 'He is just 10 years old.', 'His favorite player is Cristiano Ronaldo.']

Step 3 - Save the vectorizer in a variable

My_vect = HashingVectorizer(n_features=2**4)

Step 4 - Fit the sample text into vectorizer

Fit_text = vectorizer.fit_transform(Sample_text)

Step 5 - Print the Results

print(Fit_text, '\n') print(Fit_text.shape)

  (0, 1)	0.5
  (0, 10)	0.5
  (0, 13)	0.5
  (0, 15)	-0.5
  (1, 3)	0.5773502691896258
  (1, 7)	0.5773502691896258
  (1, 10)	0.0
  (1, 11)	0.5773502691896258
  (2, 1)	-0.4082482904638631
  (2, 3)	0.4082482904638631
  (2, 4)	-0.4082482904638631
  (2, 5)	-0.4082482904638631
  (2, 8)	-0.4082482904638631
  (2, 13)	0.4082482904638631
  (3, 0)	0.5
  (3, 2)	-0.5
  (3, 9)	-0.5
  (3, 11)	-0.5
  (3, 13)	0.0 

(4, 16)

What Users are saying..

profile image

Savvy Sahai

Data Science Intern, Capgemini
linkedin profile url

As a student looking to break into the field of data engineering and data science, one can get really confused as to which path to take. Very few ways to do it are Google, YouTube, etc. I was one of... Read More

Relevant Projects

Azure Text Analytics for Medical Search Engine Deployment
Microsoft Azure Project - Use Azure text analytics cognitive service to deploy a machine learning model into Azure Databricks

Learn to Build a Polynomial Regression Model from Scratch
In this Machine Learning Regression project, you will learn to build a polynomial regression model to predict points scored by the sports team.

House Price Prediction Project using Machine Learning in Python
Use the Zillow Zestimate Dataset to build a machine learning model for house price prediction.

Loan Default Prediction Project using Explainable AI ML Models
Loan Default Prediction Project that employs sophisticated machine learning models, such as XGBoost and Random Forest and delves deep into the realm of Explainable AI, ensuring every prediction is transparent and understandable.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Multilabel Classification Project for Predicting Shipment Modes
Multilabel Classification Project to build a machine learning model that predicts the appropriate mode of transport for each shipment, using a transport dataset with 2000 unique products. The project explores and compares four different approaches to multilabel classification, including naive independent models, classifier chains, natively multilabel models, and multilabel to multiclass approaches.

End-to-End ML Model Monitoring using Airflow and Docker
In this MLOps Project, you will learn to build an end to end pipeline to monitor any changes in the predictive power of model or degradation of data.

Expedia Hotel Recommendations Data Science Project
In this data science project, you will contextualize customer data and predict the likelihood a customer will stay at 100 different hotel groups.

Build a CNN Model with PyTorch for Image Classification
In this deep learning project, you will learn how to build an Image Classification Model using PyTorch CNN

Image Segmentation using Mask R-CNN with Tensorflow
In this Deep Learning Project on Image Segmentation Python, you will learn how to implement the Mask R-CNN model for early fire detection.