How to reduce usage of for loop by using apply in python

This recipe helps you reduce usage of for loop by using apply in python

Recipe Objective

While working with python over dataframes, for iteration, we generally use loops. However the time required is considerably high if we start using large dataset. To optimize the present problem, an other way round for the same is using apply function

So this recipe is a short example on how to reduce usage of for loop by using apply function. We will also compare time taken by apply function and with for loop. Let's get started.

Get Access to Time Series Analysis Real World Projects in Python

Step 1 - Import the library

import pandas as pd import seaborn as sb import time

Let's pause and look at these imports. Pandas is generally used for data manipulation and analysis. Here, seaborn is just taken to import some predefined datasets. Time library is generally used for controlling time while operating with commands.

Step 2 - Setup the Data

df = sb.load_dataset('tips') print(df.head())

We have imported data tips datasert from seaborn library in here.

Step 3 - Iterating using apply over total_bill column and calculating time taken in

start= time.time() df['total_bill']=df['total_bill'].apply(lambda x:x*2) print(time.time()-start) print(df.head())

In 1st line, we started our time like a stopwatch. Now using apply function, iterating over total_bill column. Now finally we print the time taken to iterate and the updated dataset.

Step 4 - Iterating using for loop over total_bill column and calculating time taken in

start=time.time() for i in range(0,len(df)): df['total_bill'][i]=df['total_bill'][i]*2 print(time.time()-start)

In 1st line, we started our time like a stopwatch. Now using for loop, iterating over total_bill column. Now finally we print the time taken to iterate and the updated dataset.

Step 5 - Lets look at our dataset now

Once we run the above code snippet, we will see:

Scroll down to the ipython notebook below to see the output.

A clear difference in time taken could be observed. For loop took around 0.5 secs while apply function took 0.007 secs.

What Users are saying..

profile image

Jingwei Li

Graduate Research assistance at Stony Brook University
linkedin profile url

ProjectPro is an awesome platform that helps me learn much hands-on industrial experience with a step-by-step walkthrough of projects. There are two primary paths to learn: Data Science and Big Data.... Read More

Relevant Projects

Build Real Estate Price Prediction Model with NLP and FastAPI
In this Real Estate Price Prediction Project, you will learn to build a real estate price prediction machine learning model and deploy it on Heroku using FastAPI Framework.

Build a Similar Images Finder with Python, Keras, and Tensorflow
Build your own image similarity application using Python to search and find images of products that are similar to any given product. You will implement the K-Nearest Neighbor algorithm to find products with maximum similarity.

AWS MLOps Project for Gaussian Process Time Series Modeling
MLOps Project to Build and Deploy a Gaussian Process Time Series Model in Python on AWS

Linear Regression Model Project in Python for Beginners Part 2
Machine Learning Linear Regression Project for Beginners in Python to Build a Multiple Linear Regression Model on Soccer Player Dataset.

NLP Project to Build a Resume Parser in Python using Spacy
Use the popular Spacy NLP python library for OCR and text classification to build a Resume Parser in Python.

Build Time Series Models for Gaussian Processes in Python
Time Series Project - A hands-on approach to Gaussian Processes for Time Series Modelling in Python

Deep Learning Project for Beginners with Source Code Part 1
Learn to implement deep neural networks in Python .

MLOps Project for a Mask R-CNN on GCP using uWSGI Flask
MLOps on GCP - Solved end-to-end MLOps Project to deploy a Mask RCNN Model for Image Segmentation as a Web Application using uWSGI Flask, Docker, and TensorFlow.

Locality Sensitive Hashing Python Code for Look-Alike Modelling
In this deep learning project, you will find similar images (lookalikes) using deep learning and locality sensitive hashing to find customers who are most likely to click on an ad.

Ola Bike Rides Request Demand Forecast
Given big data at taxi service (ride-hailing) i.e. OLA, you will learn multi-step time series forecasting and clustering with Mini-Batch K-means Algorithm on geospatial data to predict future ride requests for a particular region at a given time.