How to determine Spearmans correlation in Python?

This recipe helps you determine Spearmans correlation in Python

Recipe Objective

Spearman"s correlation is very important statical data that we need many times. We can calculate it manually but it takes time.

So this is the recipe on how we can determine Spearman"s correlation in Python

Step 1 - Importing Library

import matplotlib.pyplot as plt import scipy.stats import pandas as pd import random import seaborn as sns

We have imported stats, seaborn and pandas which is needed.

Step 2 - Creating a dataframe

We have created a empty dataframe and then added rows to it with random numbers. df = pd.DataFrame() df["x"] = random.sample(range(1, 100), 75) df["y"] = random.sample(range(1, 100), 75) print(); print(df.head())

Step 3 - Calculating Spearman"s correlation coefficient

We hawe defined a function with differnt steps that we will see.

  • We have calculated rank of x and y and passed it in the function scipy.stats.spearmanr().
  • xranks = pd.Series(xs).rank() yranks = pd.Series(ys).rank() return scipy.stats.spearmanr(xranks, yranks)
  • We have printed the result as well as the x and y values.
  • result = spearmans_rank_correlation(df.x, df.y)[0] print() print("spearmans_rank_correlation is: ", result)

Ploting Regression Plot

We are ploting regression plot with the fit. sns.lmplot("x", "y", data=df, fit_reg=True) plt.show()

   x   y
0  90  79
1  50  14
2  47  52
3  74  67
4  54  33

spearmans_rank_correlation is:  0.21755334281650068

Download Materials

What Users are saying..

profile image

Ed Godalle

Director Data Analytics at EY / EY Tech
linkedin profile url

I am the Director of Data Analytics with over 10+ years of IT experience. I have a background in SQL, Python, and Big Data working with Accenture, IBM, and Infosys. I am looking to enhance my skills... Read More

Relevant Projects

MLOps Project on GCP using Kubeflow for Model Deployment
MLOps using Kubeflow on GCP - Build and deploy a deep learning model on Google Cloud Platform using Kubeflow pipelines in Python

MLOps Project for a Mask R-CNN on GCP using uWSGI Flask
MLOps on GCP - Solved end-to-end MLOps Project to deploy a Mask RCNN Model for Image Segmentation as a Web Application using uWSGI Flask, Docker, and TensorFlow.

Build Classification Algorithms for Digital Transformation[Banking]
Implement a machine learning approach using various classification techniques in Python to examine the digitalisation process of bank customers.

Recommender System Machine Learning Project for Beginners-3
Content Based Recommender System Project - Building a Content-Based Product Recommender App with Streamlit

Time Series Forecasting Project-Building ARIMA Model in Python
Build a time series ARIMA model in Python to forecast the use of arrival rate density to support staffing decisions at call centres.

Create Your First Chatbot with RASA NLU Model and Python
Learn the basic aspects of chatbot development and open source conversational AI RASA to create a simple AI powered chatbot on your own.

Build CNN for Image Colorization using Deep Transfer Learning
Image Processing Project -Train a model for colorization to make grayscale images colorful using convolutional autoencoders.

Build Piecewise and Spline Regression Models in Python
In this Regression Project, you will learn how to build a piecewise and spline regression model from scratch in Python to predict the points scored by a sports team.

Deep Learning Project- Real-Time Fruit Detection using YOLOv4
In this deep learning project, you will learn to build an accurate, fast, and reliable real-time fruit detection system using the YOLOv4 object detection model for robotic harvesting platforms.

Many-to-One LSTM for Sentiment Analysis and Text Generation
In this LSTM Project , you will build develop a sentiment detection model using many-to-one LSTMs for accurate prediction of sentiment labels in airline text reviews. Additionally, we will also train many-to-one LSTMs on 'Alice's Adventures in Wonderland' to generate contextually relevant text.