How to create a sparse Matrix in Python?
DATA MUNGING DATA CLEANING PYTHON MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to create a sparse Matrix in Python?

How to create a sparse Matrix in Python?

This recipe helps you create a sparse Matrix in Python

3

Recipe Objective

Many a times we work on matrices in python and making Sparse Matrix manually is quite a hectic process but we know how to use python and using we can do this very well for us.

There are two popular kinds of matrices: dense and sparse. Sparse matrices have lots of 'zero' values. In machine learning projects, the learning algorithms require the data to be in-memory. If the data needed for the learning (dataframe) is not in the RAM, then the algorithm does not work. By converting a dense matrix into a sparse matrix it can be made to fit in the RAM.

There are many data structures that can be used to construct a sparse matrix in python. Python Scipy provides the following ways to represent a sparse matrix:

- Block Sparse Row matrix (BSR)
- Coordinate list matrix (COO)
- Compressed Sparse Column matrix (CSC)
- Compressed Sparse Row matrix (CSR)
- Sparse matrix with DIAgonal storage (DIA)
- Dictionary Of Keys based sparse matrix (DOK)
- Row-based linked list sparse matrix (LIL)

The recipe above takes a dense matrix and displays the various formats of sparse matrix that scipy supports.

References: https://docs.scipy.org/doc/scipy/reference/sparse.html

So this is the recipe on how we can create a sparse Matrix in Python.

Step 1 - Import the library

import numpy as np from scipy import sparse

We have imported numpy and sparse modules which will be requied.

Step 2 - Setting up the Matrix

We have created a matrix of which we will calculate sparse Matrix. matrix = np.array([[9, 8, 7], [6, 5, 4], [3, 2, 1]]) print() print("Original Matrix: \n", matrix)

Step 3 - Creating Sparse Matrices

We have created various sparse matrices by passing the original matix from the required functions.

  • Creating Dictionary Of Keys based sparse matrix (DOK).
  • print(sparse.dok_matrix(matrix))
  • Creating Block Sparse Row matrix (BSR).
  • print(sparse.bsr_matrix(matrix))
  • Creating Coordinate list matrix (COO)
  • print(sparse.coo_matrix(matrix))
  • Creating Compressed Sparse Column matrix (CSC)
  • print(sparse.csc_matrix(matrix))
  • Creating Compressed Sparse Row matrix (CSR)
  • print(sparse.csr_matrix(matrix))
  • Creating Sparse matrix with DIAgonal storage (DIA)
  • print(sparse.dia_matrix(matrix))
  • Creating Row-based linked list sparse matrix (LIL)
  • print(sparse.lil_matrix(matrix))
Now we are printing the final matrices and the output comes as:

Original Matrix: 
 [[9 8 7]
 [6 5 4]
 [3 2 1]]

Sparse Matrices: 

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (1, 0)	6
  (2, 0)	3
  (0, 1)	8
  (1, 1)	5
  (2, 1)	2
  (0, 2)	7
  (1, 2)	4
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (2, 0)	3
  (1, 0)	6
  (2, 1)	2
  (0, 0)	9
  (1, 1)	5
  (2, 2)	1
  (0, 1)	8
  (1, 2)	4
  (0, 2)	7

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

Relevant Projects

Predict Macro Economic Trends using Kaggle Financial Dataset
In this machine learning project, you will uncover the predictive value in an uncertain world by using various artificial intelligence, machine learning, advanced regression and feature transformation techniques.

Choosing the right Time Series Forecasting Methods
There are different time series forecasting methods to forecast stock price, demand etc. In this machine learning project, you will learn to determine which forecasting method to be used when and how to apply with time series forecasting example.

Walmart Sales Forecasting Data Science Project
Data Science Project in R-Predict the sales for each department using historical markdown data from the Walmart dataset containing data of 45 Walmart stores.

Data Science Project - Instacart Market Basket Analysis
Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.

Data Science Project in Python on BigMart Sales Prediction
The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.

Resume parsing with Machine learning - NLP with Python OCR and Spacy
In this machine learning resume parser example we use the popular Spacy NLP python library for OCR and text classification.

Zillow’s Home Value Prediction (Zestimate)
Data Science Project in R -Build a machine learning algorithm to predict the future sale prices of homes.

Data Science Project-TalkingData AdTracking Fraud Detection
Machine Learning Project in R-Detect fraudulent click traffic for mobile app ads using R data science programming language.

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.