How to create a sparse Matrix in Python?
DATA MUNGING DATA CLEANING PYTHON MACHINE LEARNING RECIPES PANDAS CHEATSHEET     ALL TAGS

How to create a sparse Matrix in Python?

How to create a sparse Matrix in Python?

This recipe helps you create a sparse Matrix in Python

3

Recipe Objective

Many a times we work on matrices in python and making Sparse Matrix manually is quite a hectic process but we know how to use python and using we can do this very well for us.

There are two popular kinds of matrices: dense and sparse. Sparse matrices have lots of 'zero' values. In machine learning projects, the learning algorithms require the data to be in-memory. If the data needed for the learning (dataframe) is not in the RAM, then the algorithm does not work. By converting a dense matrix into a sparse matrix it can be made to fit in the RAM.

There are many data structures that can be used to construct a sparse matrix in python. Python Scipy provides the following ways to represent a sparse matrix:

- Block Sparse Row matrix (BSR)
- Coordinate list matrix (COO)
- Compressed Sparse Column matrix (CSC)
- Compressed Sparse Row matrix (CSR)
- Sparse matrix with DIAgonal storage (DIA)
- Dictionary Of Keys based sparse matrix (DOK)
- Row-based linked list sparse matrix (LIL)

The recipe above takes a dense matrix and displays the various formats of sparse matrix that scipy supports.

References: https://docs.scipy.org/doc/scipy/reference/sparse.html

So this is the recipe on how we can create a sparse Matrix in Python.

Step 1 - Import the library

import numpy as np from scipy import sparse

We have imported numpy and sparse modules which will be requied.

Step 2 - Setting up the Matrix

We have created a matrix of which we will calculate sparse Matrix. matrix = np.array([[9, 8, 7], [6, 5, 4], [3, 2, 1]]) print() print("Original Matrix: \n", matrix)

Step 3 - Creating Sparse Matrices

We have created various sparse matrices by passing the original matix from the required functions.

  • Creating Dictionary Of Keys based sparse matrix (DOK).
  • print(sparse.dok_matrix(matrix))
  • Creating Block Sparse Row matrix (BSR).
  • print(sparse.bsr_matrix(matrix))
  • Creating Coordinate list matrix (COO)
  • print(sparse.coo_matrix(matrix))
  • Creating Compressed Sparse Column matrix (CSC)
  • print(sparse.csc_matrix(matrix))
  • Creating Compressed Sparse Row matrix (CSR)
  • print(sparse.csr_matrix(matrix))
  • Creating Sparse matrix with DIAgonal storage (DIA)
  • print(sparse.dia_matrix(matrix))
  • Creating Row-based linked list sparse matrix (LIL)
  • print(sparse.lil_matrix(matrix))
Now we are printing the final matrices and the output comes as:

Original Matrix: 
 [[9 8 7]
 [6 5 4]
 [3 2 1]]

Sparse Matrices: 

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (0, 0)	9
  (1, 0)	6
  (2, 0)	3
  (0, 1)	8
  (1, 1)	5
  (2, 1)	2
  (0, 2)	7
  (1, 2)	4
  (2, 2)	1

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

  (2, 0)	3
  (1, 0)	6
  (2, 1)	2
  (0, 0)	9
  (1, 1)	5
  (2, 2)	1
  (0, 1)	8
  (1, 2)	4
  (0, 2)	7

  (0, 0)	9
  (0, 1)	8
  (0, 2)	7
  (1, 0)	6
  (1, 1)	5
  (1, 2)	4
  (2, 0)	3
  (2, 1)	2
  (2, 2)	1

Relevant Projects

Zillow’s Home Value Prediction (Zestimate)
Data Science Project in R -Build a machine learning algorithm to predict the future sale prices of homes.

Data Science Project in Python on BigMart Sales Prediction
The goal of this data science project is to build a predictive model and find out the sales of each product at a given Big Mart store.

Deep Learning with Keras in R to Predict Customer Churn
In this deep learning project, we will predict customer churn using Artificial Neural Networks and learn how to model an ANN in R with the keras deep learning package.

Data Science Project-TalkingData AdTracking Fraud Detection
Machine Learning Project in R-Detect fraudulent click traffic for mobile app ads using R data science programming language.

Machine Learning project for Retail Price Optimization
In this machine learning pricing project, we implement a retail price optimization algorithm using regression trees. This is one of the first steps to building a dynamic pricing model.

Build a Collaborative Filtering Recommender System in Python
Use the Amazon Reviews/Ratings dataset of 2 Million records to build a recommender system using memory-based collaborative filtering in Python.

Sequence Classification with LSTM RNN in Python with Keras
In this project, we are going to work on Sequence to Sequence Prediction using IMDB Movie Review Dataset​ using Keras in Python.

Identifying Product Bundles from Sales Data Using R Language
In this data science project in R, we are going to talk about subjective segmentation which is a clustering technique to find out product bundles in sales data.

Predict Census Income using Deep Learning Models
In this project, we are going to work on Deep Learning using H2O to predict Census income.

Build an Image Classifier for Plant Species Identification
In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques.