How to find common elements between 2 pandas dataframes?

How to find common elements between 2 pandas dataframes?

How to find common elements between 2 pandas dataframes?

This recipe helps you find common elements between 2 pandas dataframes


Recipe Objective

While working with dataframes, many a times we have two dataframes and there is a need to find common elements between them. Such problems can be easily handled by merge function.

So this recipe is a short example on how to find common elements between 2 pandas dataframes. Let's get started.

Step 1 - Import the library

import pandas as pd

Let's pause and look at these imports. Pandas is generally used for data manipulation and analysis.

Step 2 - Setup the Data

df1= pd.DataFrame({'Student': ['Ram','Rohan','Shyam','Mohan'], 'Grade': ['A','C','B','Ex']}) df2 = pd.DataFrame({'Student': ['Ram','Shyam','Raunak'], 'Grade': ['A','B','F']})

Let us create a two simple dataset of Student and grades.

Step 3 - Finding Common Elements

df3=pd.merge(df1,df2, how='inner')

Merge function in pandas library help us in performing all types of binary operation over dataframes. Here we simply merge both on all columns keep 'how' as 'inner' which simply means finding common elements.

Step 4 - Printing results

print('df1\n',df1) print('df2\n',df2) print('df1 union df2\n',df3)

Simply use print function to print df1, df2 and our new dataframe df1 union df2

Step 5 - Let's look at our dataset now

Once we run the above code snippet, we will see:

Scroll down to the ipython notebook below to see the output.

Relevant Projects

Zillow’s Home Value Prediction (Zestimate)
Data Science Project in R -Build a machine learning algorithm to predict the future sale prices of homes.

Music Recommendation System Project using Python and R
Machine Learning Project - Work with KKBOX's Music Recommendation System dataset to build the best music recommendation engine.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

Resume parsing with Machine learning - NLP with Python OCR and Spacy
In this machine learning resume parser example we use the popular Spacy NLP python library for OCR and text classification.

Data Science Project - Instacart Market Basket Analysis
Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.

Solving Multiple Classification use cases Using H2O
In this project, we are going to talk about H2O and functionality in terms of building Machine Learning models.

Data Science Project in Python on BigMart Sales Prediction
The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.

Build a Collaborative Filtering Recommender System in Python
Use the Amazon Reviews/Ratings dataset of 2 Million records to build a recommender system using memory-based collaborative filtering in Python.

Perform Time series modelling using Facebook Prophet
In this project, we are going to talk about Time Series Forecasting to predict the electricity requirement for a particular house using Prophet.

Identifying Product Bundles from Sales Data Using R Language
In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.