How to reduce usage of for loop by using apply in python

This recipe helps you reduce usage of for loop by using apply in python

Recipe Objective

While working with python over dataframes, for iteration, we generally use loops. However the time required is considerably high if we start using large dataset. To optimize the present problem, an other way round for the same is using apply function

So this recipe is a short example on how to reduce usage of for loop by using apply function. We will also compare time taken by apply function and with for loop. Let's get started.

Get Access to Time Series Analysis Real World Projects in Python

Step 1 - Import the library

import pandas as pd import seaborn as sb import time

Let's pause and look at these imports. Pandas is generally used for data manipulation and analysis. Here, seaborn is just taken to import some predefined datasets. Time library is generally used for controlling time while operating with commands.

Step 2 - Setup the Data

df = sb.load_dataset('tips') print(df.head())

We have imported data tips datasert from seaborn library in here.

Step 3 - Iterating using apply over total_bill column and calculating time taken in

start= time.time() df['total_bill']=df['total_bill'].apply(lambda x:x*2) print(time.time()-start) print(df.head())

In 1st line, we started our time like a stopwatch. Now using apply function, iterating over total_bill column. Now finally we print the time taken to iterate and the updated dataset.

Step 4 - Iterating using for loop over total_bill column and calculating time taken in

start=time.time() for i in range(0,len(df)): df['total_bill'][i]=df['total_bill'][i]*2 print(time.time()-start)

In 1st line, we started our time like a stopwatch. Now using for loop, iterating over total_bill column. Now finally we print the time taken to iterate and the updated dataset.

Step 5 - Lets look at our dataset now

Once we run the above code snippet, we will see:

Scroll down to the ipython notebook below to see the output.

A clear difference in time taken could be observed. For loop took around 0.5 secs while apply function took 0.007 secs.

What Users are saying..

profile image

Ed Godalle

Director Data Analytics at EY / EY Tech
linkedin profile url

I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More

Relevant Projects

Time Series Python Project using Greykite and Neural Prophet
In this time series project, you will forecast Walmart sales over time using the powerful, fast, and flexible time series forecasting library Greykite that helps automate time series problems.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

Customer Market Basket Analysis using Apriori and Fpgrowth algorithms
In this data science project, you will learn how to perform market basket analysis with the application of Apriori and FP growth algorithms based on the concept of association rule learning.

Learn How to Build PyTorch Neural Networks from Scratch
In this deep learning project, you will learn how to build PyTorch neural networks from scratch.

Locality Sensitive Hashing Python Code for Look-Alike Modelling
In this deep learning project, you will find similar images (lookalikes) using deep learning and locality sensitive hashing to find customers who are most likely to click on an ad.

Build an optimal End-to-End MLOps Pipeline and Deploy on GCP
Learn how to build and deploy an end-to-end optimal MLOps Pipeline for Loan Eligibility Prediction Model in Python on GCP

Hands-On Approach to Causal Inference in Machine Learning
In this Machine Learning Project, you will learn to implement various causal inference techniques in Python to determine, how effective the sprinkler is in making the grass wet.

Stock Price Prediction Project using LSTM and RNN
Learn how to predict stock prices using RNN and LSTM models. Understand deep learning concepts and apply them to real-world financial data for accurate forecasting.

MLOps Project to Build Search Relevancy Algorithm with SBERT
In this MLOps SBERT project you will learn to build and deploy an accurate and scalable search algorithm on AWS using SBERT and ANNOY to enhance search relevancy in news articles.

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .