Each project comes with 2-5 hours of micro-videos explaining the solution.
Code & Dataset
Get access to 50+ solved projects with iPython notebooks and datasets.
Add project experience to your Linkedin/Github profiles.
Understanding the problem statement
Initializing necessary libraries and understanding its use
Difference between NLU and NLG
Constructing features that we will later use to embed our samples with
Visualizing distribution of Target variable using Barplot
Plotting Log Histogram
Visualizing distribution of corpus using Barplot
Normalized histogram of character count in questions
t-Distributed Stochastic Neighbor Embedding for dimensionality reduction
Stemming, Lemmatization and removing stopwords
What is Tokenization,N-grams and POS tagging
Converting Word to Vectors and making it ready for training
Calculating Cosine Similarity between different words
Applying Logistic Regression
Applying ensemble model Random Forest Classifier
Applying Gradient Boosting Classifier
Applying Grid search CV for hyperparameter tuning
Defining F1 metrics and Confusion Metrics
Modularizing the code using Pipeline
Where else but Quora can a physicist helps a chef with a math problem and gets cooking tips in return? Quora is a place to gain and share knowledge—about anything. It’s a platform to ask questions and connect with people who contribute unique insights and quality answers. This empowers people to learn from each other and to better understand the world.
Over 100 million people visit Quora every month, so it's no surprise that many people ask similarly worded questions. Multiple questions with the same intent can cause seekers to spend more time finding the best answer to their question and make writers feel they need to answer multiple versions of the same question. Quora values canonical questions because they provide a better experience to active seekers and writers, and offer more value to both of these groups in the long term.
Currently, Quora uses a Random Forest model to identify duplicate questions. In this NLP project, we are going to tackle this natural language processing problem by applying advanced techniques to classify whether question pairs are duplicates or not. Doing so will make it easier to find high-quality answers to questions resulting in an improved experience for Quora writers, seekers, and readers.
The goal of this NLP project in Python is to predict which of the provided pairs of questions contain two questions with the same meaning. The ground truth is the set of labels that have been supplied by human experts. The ground truth labels are inherently subjective, as the true meaning of sentences can never be known with certainty. Human labeling is also a 'noisy' process, and reasonable people will disagree. As a result, the ground truth labels on this dataset should be taken to be 'informed' but not 100% accurate, and may include incorrect labeling. We believe the labels, on the whole, to represent a reasonable consensus, but this may often not be true on a case by case basis for individual items in the dataset.