How to use for loop with dask for parallel processing?
This recipe helps you use for loop with dask for parallel processing
Recipe Objective
How to use for loop with dask for parallel processing.
We will transform the function inc to be used parallely with the help of dask delayed, and we will show you the visualizaion of the for loop to understand parallel working of it.
Table of Contents
Step 1- Importing Libraries.
from dask import delayed
Step 2- Defining a function.
We will define a incremental function which will increase the value by 2 in a for loop.
def inc(x):
return x + 2
Step 3- Applying delayed.
We will apply dask delayed in the for loop to run inc function parallely.
data = [1, 2, 3, 4, 5]
results = []
for x in data:
y = delayed(inc)(x)
results.append(y)
Step 4- Displaying results.
We will print the results of calculating addition and final computing.
addition = delayed(sum)(results)
print("Before addition:", addition)
calculate = addition.compute()
print("After calculating :", calculate)
Step 5- Visualizing
Visualizing the whole addition process that was done in parallel.
addition.visualize()
What Users are saying..
Anand Kumpatla
ProjectPro is a unique platform and helps many people in the industry to solve real-life problems with a step-by-step walkthrough of projects. A platform with some fantastic resources to gain... Read More