How to create a sparse Matrix in Python?
DATA MUNGING DATA CLEANING PYTHON MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to create a sparse Matrix in Python?

How to create a sparse Matrix in Python?

This recipe helps you create a sparse Matrix in Python

3

Recipe Objective

Many a times we work on matrices in python and making Sparse Matrix manually is quite a hectic process but we know how to use python and using we can do this very well for us.

There are two popular kinds of matrices: dense and sparse. Sparse matrices have lots of 'zero' values. In machine learning projects, the learning algorithms require the data to be in-memory. If the data needed for the learning (dataframe) is not in the RAM, then the algorithm does not work. By converting a dense matrix into a sparse matrix it can be made to fit in the RAM.

There are many data structures that can be used to construct a sparse matrix in python. Python Scipy provides the following ways to represent a sparse matrix:

- Block Sparse Row matrix (BSR)
- Coordinate list matrix (COO)
- Compressed Sparse Column matrix (CSC)
- Compressed Sparse Row matrix (CSR)
- Sparse matrix with DIAgonal storage (DIA)
- Dictionary Of Keys based sparse matrix (DOK)
- Row-based linked list sparse matrix (LIL)

The recipe above takes a dense matrix and displays the various formats of sparse matrix that scipy supports.

References: https://docs.scipy.org/doc/scipy/reference/sparse.html

So this is the recipe on how we can create a sparse Matrix in Python.

Step 1 - Import the library

import numpy as np from scipy import sparse

We have imported numpy and sparse modules which will be requied.

Step 2 - Setting up the Matrix

We have created a matrix of which we will calculate sparse Matrix. matrix = np.array([[9, 8, 7], [6, 5, 4], [3, 2, 1]]) print() print("Original Matrix: \n", matrix)

Step 3 - Creating Sparse Matrices

We have created various sparse matrices by passing the original matix from the required functions.

  • Creating Dictionary Of Keys based sparse matrix (DOK).
  • print(sparse.dok_matrix(matrix))
  • Creating Block Sparse Row matrix (BSR).
  • print(sparse.bsr_matrix(matrix))
  • Creating Coordinate list matrix (COO)
  • print(sparse.coo_matrix(matrix))
  • Creating Compressed Sparse Column matrix (CSC)
  • print(sparse.csc_matrix(matrix))
  • Creating Compressed Sparse Row matrix (CSR)
  • print(sparse.csr_matrix(matrix))
  • Creating Sparse matrix with DIAgonal storage (DIA)
  • print(sparse.dia_matrix(matrix))
  • Creating Row-based linked list sparse matrix (LIL)
  • print(sparse.lil_matrix(matrix))
Now we are printing the final matrices and the output comes as:

Original Matrix: 
 [[9 8 7]
 [6 5 4]
 [3 2 1]]

Sparse Matrices: 

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (1, 0)	6
  (2, 0)	3
  (0, 1)	8
  (1, 1)	5
  (2, 1)	2
  (0, 2)	7
  (1, 2)	4
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (2, 0)	3
  (1, 0)	6
  (2, 1)	2
  (0, 0)	9
  (1, 1)	5
  (2, 2)	1
  (0, 1)	8
  (1, 2)	4
  (0, 2)	7

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

Relevant Projects

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.

Credit Card Fraud Detection as a Classification Problem
In this data science project, we will predict the credit card fraud in the transactional dataset using some of the predictive models.

Human Activity Recognition Using Smartphones Data Set
In this deep learning project, you will build a classification system where to precisely identify human fitness activities.

PySpark Tutorial - Learn to use Apache Spark with Python
PySpark Project-Get a handle on using Python with Spark through this hands-on data processing spark python tutorial.

Solving Multiple Classification use cases Using H2O
In this project, we are going to talk about H2O and functionality in terms of building Machine Learning models.

Predict Credit Default | Give Me Some Credit Kaggle
In this data science project, you will predict borrowers chance of defaulting on credit loans by building a credit score prediction model.

Ensemble Machine Learning Project - All State Insurance Claims Severity Prediction
In this ensemble machine learning project, we will predict what kind of claims an insurance company will get. This is implemented in python using ensemble machine learning algorithms.

Loan Eligibility Prediction using Gradient Boosting Classifier
This data science in python project predicts if a loan should be given to an applicant or not. We predict if the customer is eligible for loan based on several factors like credit score and past history.

Resume parsing with Machine learning - NLP with Python OCR and Spacy
In this machine learning resume parser example we use the popular Spacy NLP python library for OCR and text classification.

Data Science Project - Instacart Market Basket Analysis
Data Science Project - Build a recommendation engine which will predict the products to be purchased by an Instacart consumer again.