How to perform logistic regression using tensorflow

This recipe helps you perform logistic regression using tensorflow

Recipe Objective

How to perform logistic regression using tensorflow?

This is the classification algorithm which is mostly used in Machine learning, the algorithm allows the data to be categorized in the discrete classes by learning the relationship from a given set of the labeled data.

Complete Guide to Tensorflow for Deep Learning with Python for Free

Step 1 - Import library

import numpy as np import pandas as pd import tensorflow as tf import matplotlib.pyplot as plt from sklearn.preprocessing import OneHotEncoder

Step 2 - Load the data

data_iris = pd.read_csv('/content/dataset.csv', header = None) print("This is shape of our data:", data_iris.shape) print("This is the first five rows of our data:","\n",data_iris.head())

This is shape of our data: (100, 4)
This is the first five rows of our data: 
    0    1    2  3
0  0  5.1  3.5  1
1  1  4.9  3.0  1
2  2  4.7  3.2  1
3  3  4.6  3.1  1
4  4  5.0  3.6  1 

Step 3 - Create feature matrix and data lables

x_data = data_iris.iloc[:, 1:-1].values y_data = data_iris.iloc[:, -1:].values print("This is the Shape of Feature Matrix:", x_data.shape) print("This is the Shape Label Vector:", y_data.shape)

This is the Shape of Feature Matrix: (100, 2)
This is the Shape Label Vector: (100, 1)

Step 4 - Visualize the original data

positive_x_data = np.array([x_data[i] for i in range(len(x_data)) if y_data[i] == 1]) # These are the Positive Data Points negative_x_data = np.array([x_data[i] for i in range(len(x_data)) if y_data[i] == 0]) # These are the Negative Data Points plt.scatter(positive_x_data[:, 0], positive_x_data[:, 1], color = 'green', label = 'This is Positive data') # Plotting the Positive Data Points plt.scatter(negative_x_data[:, 0], negative_x_data[:, 1], color = 'red', label = 'This is Negative data') # Plotting the Negative Data Points plt.xlabel('data for feature one') plt.ylabel('data for feature two') plt.title('Results for given data') plt.legend() plt.show()

Step 5 - Create one hot encoder

encoder_oneHot = OneHotEncoder() #Create the one hot encoder encoder_oneHot.fit(x_data) x_new = encoder_oneHot.transform(x_data).toarray() #this is encoding for x_new data encoder_oneHot.fit(y_data) y_new = encoder_oneHot.transform(y_data).toarray() #this is encoding for y_new data rate, epochs = 0.0035, 500 m, n = x_new.shape print('m =', m) print('n =', n) print('Defined Learning Rate =', rate) print('Defined Number of Epochs =', epochs)

m = 100
n = 51
Defined Learning Rate = 0.0035
Defined Number of Epochs = 500

Here we are creating the one hot encoder, which works better with classification and the regression algorithms. The one hot encoder will transform the categorical columns or features which work fine with the algorithms. After we have also set the learning rate and the number of epochs.

Step 6 - Start creating model

X_tensor = tf.compat.v1.placeholder(tf.float32, [None, n]) # There are n columns in the feature matrix after One Hot Encoding. Y_tensor = tf.compat.v1.placeholder(tf.float32, [None, 2]) # Since this is a binary classification problem, Y_tensor can take only 2 values. Weight_data = tf.Variable(tf.zeros([n, 2])) # Trainable Variable Weights bias_data = tf.Variable(tf.zeros([2])) # Trainable Variable Bias

Here we started creating our model by defining the placeholders which are X_tensor and Y_tensor. So these X and Y which are our training example can feed into the optimizer during the training process, also the trainable variables is being created as Weights_data and bias_data which can be optimized by the gradient decent optimizer

Step 7 - Declare the functions

Y_hat_data = tf.nn.sigmoid(tf.add(tf.matmul(X_tensor, Weight_data), bias_data)) # This is the Hypothesis data cost_function = tf.nn.sigmoid_cross_entropy_with_logits(logits = Y_hat_data, labels = Y_tensor) # This is the Sigmoid Cross Entropy Cost Function gradient_optimizer = tf.compat.v1.train.GradientDescentOptimizer(learning_rate = rate).minimize(cost_function) # This is the Gradient Descent Optimizer initialize = tf.compat.v1.global_variables_initializer() # This is the Global Var

Here in the above we are declaring the hypothesis as Y_hat_data, cost_function, optimizer and the Global variable Initializer.

Step 8 - Start Training

with tf.compat.v1.Session() as sess: # Starting the Tensorflow Session # This will Initialize the Variables sess.run(initialize) # Lists for storing the changing Cost and Accuracy in every Epoch cost_history_data, accuracy_history_data = [], [] # Iterating through all the epochs for epoch in range(epochs): cost_per_epoch = 0 # Running the Optimizer sess.run(gradient_optimizer, feed_dict = {X_tensor : x_new, Y_tensor : y_new}) # Calculating cost on current Epoch cal_cost = sess.run(cost_function, feed_dict = {X_tensor : x_new, Y_tensor : y_new}) # Calculating accuracy on current Epoch Make_prediction = tf.equal(tf.argmax(Y_hat_data, 1), tf.argmax(Y_tensor, 1)) accuracy = tf.reduce_mean(tf.cast(Make_prediction, tf.float32)) # Storing Cost and Accuracy to the history cost_history_data.append(sum(sum(cal_cost))) accuracy_history_data.append(accuracy.eval({X_tensor : x_new, Y_tensor : y_new}) * 100) # Displaying result on current Epoch if epoch % 100 == 0 and epoch != 0: print("Epoch " + str(epoch) + " Cost: " + str(cost_history_data[-1])) Weight = sess.run(Weight_data) # Optimized Weight Bias = sess.run(bias_data) # Optimized Bias # Final Accuracy Make_prediction = tf.equal(tf.argmax(Y_hat_data, 1), tf.argmax(Y_tensor, 1)) accuracy = tf.reduce_mean(tf.cast(Make_prediction, tf.float32)) print("\nAccuracy:", accuracy_history_data[-1], "%")

Epoch 100 Cost: 136.33413696289062
Epoch 200 Cost: 132.68544006347656
Epoch 300 Cost: 129.77124786376953
Epoch 400 Cost: 127.2939567565918

Accuracy: 87.00000047683716 %

Step 9 - Visualize Cost over epochs

plt.plot(list(range(epochs)), cost_history_data) plt.xlabel('Epochs data') plt.ylabel('Cost data') plt.title('This is the Decrease in Cost with Epochs') plt.show()

Step 10 - Visualize Accuracy over epochs

plt.plot(list(range(epochs)), accuracy_history_data) plt.xlabel('Epochs data') plt.ylabel('Accuracy data') plt.title('This is the Increase in Accuracy with Epochs') plt.show() {"mode":"full","isActive":false}

What Users are saying..

profile image

Savvy Sahai

Data Science Intern, Capgemini
linkedin profile url

As a student looking to break into the field of data engineering and data science, one can get really confused as to which path to take. Very few ways to do it are Google, YouTube, etc. I was one of... Read More

Relevant Projects

Create Your First Chatbot with RASA NLU Model and Python
Learn the basic aspects of chatbot development and open source conversational AI RASA to create a simple AI powered chatbot on your own.

Build a Multi-Class Classification Model in Python on Saturn Cloud
In this machine learning classification project, you will build a multi-class classification model in Python on Saturn Cloud to predict the license status of a business.

Build a Multi Class Image Classification Model Python using CNN
This project explains How to build a Sequential Model that can perform Multi Class Image Classification in Python using CNN

Linear Regression Model Project in Python for Beginners Part 1
Machine Learning Linear Regression Project in Python to build a simple linear regression model and master the fundamentals of regression for beginners.

Linear Regression Model Project in Python for Beginners Part 2
Machine Learning Linear Regression Project for Beginners in Python to Build a Multiple Linear Regression Model on Soccer Player Dataset.

Llama2 Project for MetaData Generation using FAISS and RAGs
In this LLM Llama2 Project, you will automate metadata generation using Llama2, RAGs, and AWS to reduce manual efforts.

Learn Object Tracking (SOT, MOT) using OpenCV and Python
Get Started with Object Tracking using OpenCV and Python - Learn to implement Multiple Instance Learning Tracker (MIL) algorithm, Generic Object Tracking Using Regression Networks Tracker (GOTURN) algorithm, Kernelized Correlation Filters Tracker (KCF) algorithm, Tracking, Learning, Detection Tracker (TLD) algorithm for single and multiple object tracking from various video clips.

End-to-End Snowflake Healthcare Analytics Project on AWS-2
In this AWS Snowflake project, you will build an end to end retraining pipeline by checking Data and Model Drift and learn how to redeploy the model if needed

MLOps Project on GCP using Kubeflow for Model Deployment
MLOps using Kubeflow on GCP - Build and deploy a deep learning model on Google Cloud Platform using Kubeflow pipelines in Python

MLOps Project to Build Search Relevancy Algorithm with SBERT
In this MLOps SBERT project you will learn to build and deploy an accurate and scalable search algorithm on AWS using SBERT and ANNOY to enhance search relevancy in news articles.