While working with python over dataframes, for iteration, we generally use loops. However the time required is considerably high if we start using large dataset. To optimize the present problem, an other way round for the same is using apply function
So this recipe is a short example on how to reduce usage of for loop by using apply function. We will also compare time taken by apply function and with for loop. Let's get started.
import pandas as pd import seaborn as sb import time
Let's pause and look at these imports. Pandas is generally used for data manipulation and analysis. Here, seaborn is just taken to import some predefined datasets. Time library is generally used for controlling time while operating with commands.
df = sb.load_dataset('tips') print(df.head())
We have imported data tips datasert from seaborn library in here.
start= time.time() df['total_bill']=df['total_bill'].apply(lambda x:x*2) print(time.time()-start) print(df.head())
In 1st line, we started our time like a stopwatch. Now using apply function, iterating over total_bill column. Now finally we print the time taken to iterate and the updated dataset.
start=time.time() for i in range(0,len(df)): df['total_bill'][i]=df['total_bill'][i]*2 print(time.time()-start)
In 1st line, we started our time like a stopwatch. Now using for loop, iterating over total_bill column. Now finally we print the time taken to iterate and the updated dataset.
Once we run the above code snippet, we will see:
Scroll down to the ipython notebook below to see the output.
A clear difference in time taken could be observed. For loop took around 0.5 secs while apply function took 0.007 secs.