What are lazy functions and how does dask deal with them?
MACHINE LEARNING RECIPES DATA CLEANING PYTHON DATA MUNGING PANDAS CHEATSHEET     ALL TAGS

What are lazy functions and how does dask deal with them?

What are lazy functions and how does dask deal with them?

This recipe explains what are lazy functions and how does dask deal with them

0

Recipe Objective

What are lazy functions? How does dask deal with them.

In python we have something called lazy functions specially for big DataFrames we call it lazy execution of code.

Sometimes problems don't fit in the single code, or the RAM could not hold the long execution of code, sometimes dask.arrays or dask.dataframe fails to manage the long Datasets. To overcome those problems we can use dask.delayed interface in this the users can parallelize the custom algorithms.It allows users to create graphs directly with a light annotation of normal python code.

Step 1- Importing Libraries.

import dask

Step 2- Creating a Dask Delayed code for sample.

We will define some sample functions and then we will apply the Delay function after combining all the functions.

def increase(x): return x + 2 def triple(x): return x * 3 def divide(x,y): return y/x def add(x, y, z): return x + y +z

Step 3- Creating a sample list.

Now we will pass the list from every defined function and compute the final data.

data = [5, 10, 15, 20, 25, 30, 35, 40] y=100 output = [] for x in data: a = increase(x) b = triple(x) c = divide(x,y) d = add(a, b, c) output.append(d) total = dask.delayed(sum)(output) total

Relevant Projects

German Credit Dataset Analysis to Classify Loan Applications
In this data science project, you will work with German credit dataset using classification techniques like Decision Tree, Neural Networks etc to classify loan applications using R.

Machine Learning or Predictive Models in IoT - Energy Prediction Use Case
In this machine learning and IoT project, we are going to test out the experimental data using various predictive models and train the models and break the energy usage.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Deep Learning with Keras in R to Predict Customer Churn
In this deep learning project, we will predict customer churn using Artificial Neural Networks and learn how to model an ANN in R with the keras deep learning package.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Customer Churn Prediction Analysis using Ensemble Techniques
In this machine learning churn project, we implement a churn prediction model in python using ensemble techniques.

NLP and Deep Learning For Fake News Classification in Python
In this project you will use Python to implement various machine learning methods( RNN, LSTM, GRU) for fake news classification.

Human Activity Recognition Using Multiclass Classification in Python
In this human activity recognition project, we use multiclass classification machine learning techniques to analyse fitness dataset from a smartphone tracker.

Data Science Project in Python on BigMart Sales Prediction
The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.

Data Science Project - Instacart Market Basket Analysis
Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.