How to segregate duplicate values from Pandas dataframe?
This recipe helps you segregate duplicate values from Pandas dataframe
Recipe Objective
Suppose we have duplicate data in our dataset. Now its best to segregate and remove them.
So this recipe is a short example on How to segregate duplicate values from Pandas dataframe. Let's get started.
Step 1 - Import the library
import pandas as pd
Let's pause and look at these imports. Pandas is generally used for performing mathematical operation and preferably over arrays.
Step 2 - Setup the Data
df = pd.DataFrame({"A":[0, 1, 2, 3, 5, 9],
"B":[11, 5, 8, 6, 7, 8],
"C":[2, 5, 10, 11, 9, 8]})
Here we have setup a random dataset with some random values in it.
Step 3 - Segregating out duplicates
print(df['A'])
print(set(df['A']))
Here we are our original column having duplicate values. Now using set function, we have simply segregated and dropped duplicate values.
Step 4 - Let's look at our dataset now
Once we run the above code snippet, we will see:
Scroll down to the ipython file to look at the results.
We can see the duplicate value 5 getting dropped out from final results. This operation will remain consistent even with strings.
What Users are saying..
Savvy Sahai
As a student looking to break into the field of data engineering and data science, one can get really confused as to which path to take. Very few ways to do it are Google, YouTube, etc. I was one of... Read More