How to segregate duplicate values from Pandas dataframe?
This recipe helps you segregate duplicate values from Pandas dataframe
Recipe Objective
Suppose we have duplicate data in our dataset. Now its best to segregate and remove them.
So this recipe is a short example on How to segregate duplicate values from Pandas dataframe. Let's get started.
Step 1 - Import the library
import pandas as pd
Let's pause and look at these imports. Pandas is generally used for performing mathematical operation and preferably over arrays.
Step 2 - Setup the Data
df = pd.DataFrame({"A":[0, 1, 2, 3, 5, 9],
"B":[11, 5, 8, 6, 7, 8],
"C":[2, 5, 10, 11, 9, 8]})
Here we have setup a random dataset with some random values in it.
Step 3 - Segregating out duplicates
print(df['A'])
print(set(df['A']))
Here we are our original column having duplicate values. Now using set function, we have simply segregated and dropped duplicate values.
Step 4 - Let's look at our dataset now
Once we run the above code snippet, we will see:
Scroll down to the ipython file to look at the results.
We can see the duplicate value 5 getting dropped out from final results. This operation will remain consistent even with strings.
What Users are saying..
Ed Godalle
I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More