<jupyter_start><jupyter_text>**Logistic Regresison**It is generally used for categorical classification like if a person is veg/non-veg or classifiying an email as spam or not spam.It has the same hypothesis as that of the linear regression except for the fact that it uses the logistic function. Logistic function is a function of a variable x having a S-shaped curve and is given by ![alt text](https://drive.google.com/uc?id=1lyWmjy9j4cxn1_Z0qsKzDULXb4m8XCfQ) where L,k,x0 are constantsThe sigmoid function is used as the logistic function. Sigmoid funcion is given by ![alt text](https://drive.google.com/uc?id=1v3LiP4rc2fWZgp6AiCdRUr_h_Ig1S18Q)What sigmoid function does is basically it squeezes the value of input which can lie in the range of real numbers to the range (0,1).<jupyter_code>#import necessary libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt                  #Matplotlib is used for plotting grpahs, histograms, etc.

from sklearn.linear_model import LogisticRegression         #sklearn contians all pre-defined models, like linear regression, logistic regresion,etc.
#Mount google drive on colab. The dataset is uploaded on Google Drive 

from google.colab import drive          
drive.mount('/content/drive')
#read the data from the path specified. We have taken the very famous Iris Data Set.
#read our dataset into a dataframe labelled data with the help of pandas. 
#We copy paste the path where our dataset was present in the drive.
#Since our data is in a csv file we use pd.read_csv

data = pd.read_csv('/content/drive/My Drive/Iris Data Set.csv')
#We display the data 
data.head(150)
#plot the graph

plt.scatter(data['SepalLengthCm'][:50],data['SepalWidthCm'][:50])                          #plotting points on a graph 
plt.scatter(data['SepalLengthCm'][50:100],data['SepalWidthCm'][50:100],color='red')        
plt.scatter(data['SepalLengthCm'][100:],data['SepalWidthCm'][100:],color='green')        
plt.xlabel("Sepal Length in Cm")                                                            #Labelling the x and y axis
plt.ylabel("Sepal Width in Cm")
plt.show() 

plt.scatter(data['PetalLengthCm'][:50],data['PetalWidthCm'][:50])                          #plotting points on a graph 
plt.scatter(data['PetalLengthCm'][50:100],data['PetalWidthCm'][50:100],color='red')        
plt.scatter(data['PetalLengthCm'][100:],data['PetalWidthCm'][100:],color='green')        
plt.xlabel("Petal Length in Cm")                                                            #Labelling the x and y axis
plt.ylabel("Petal Width in Cm")
plt.show() 

plt.scatter(data['PetalLengthCm'][:50],data['PetalWidthCm'][:50])                          #plotting points on a graph 
plt.scatter(data['PetalLengthCm'][50:100],data['PetalWidthCm'][50:100],color='red')              
plt.xlabel("Petal Length in Cm")                                                            #Labelling the x and y axis
plt.ylabel("Petal Width in Cm")
plt.show() 

#For classification we rename the species of the flowers as 0,1,2

data["Species"].replace({"Iris-setosa":0,"Iris-versicolor":1, "Iris-virginica":2}, inplace=True)
data.head(150)<jupyter_output><empty_output><jupyter_text>For simplicity we try to classify only the Iris-setosa and the Iris-versicolor on the basis of their sepal width only. For simplicity we have taken only one feature. Here we have used Logistic regression for binary classification.<jupyter_code>#we plot the data for sepal widths of the two classes taken into consideration. 
#For classifiaction and regression purposes we have used the integers as species

plt.scatter(data['SepalWidthCm'][:50],data['Species'][:50],color='blue')                          #plotting points on a graph 
plt.scatter(data['SepalWidthCm'][50:100],data['Species'][50:100],color='red')               
plt.xlabel("Sepal Width in Cm")                                                            #Labelling the x and y axis
plt.ylabel("Species")
plt.show() 
#We reshape the vector as when we read the data from csv we read it as a vector. 
#We reshape the vector so that we get a vector of 1 dimension only 

X = data['SepalWidthCm'].values.reshape(-1,1)
Y = data['Species'].values.reshape(-1,1)<jupyter_output><empty_output><jupyter_text>Cost function is given by  ![alt text](https://drive.google.com/uc?id=1h52Pca9WVhCxI9h9KrUxAfefiSL3KE5i)It can also be written as J(m) = - [ylog(p) + (1-y)log(1-p)]This is written taking into consideration that m is a single scalar quantity. However usually, m is a vector quantity and then we take the sum of all the vector values and average it out. So the cost function becomes ![alt text](https://drive.google.com/uc?id=1OZqgu2tG6m8uAFtJhmPeaZKOndckWSMZ)But here we have used our normal mean squared error loss just for simplicity !<jupyter_code>#initialize m and c randomly. m and c are the weights  
m=3
c=0
alpha=0.01            
epochs = 10000
e=2.71828

#We define the sigmoid function 
def sigmoid(n):
  return 1/(1+ e**(-n))

#we define the Logistic Regression Function as given 
#In the function, np is our probability hypothesis. For the probability to be between 0 and 1 we used sigmoid function. m is the weight assumed.
#So hp gives us the probability that y=1 on giving the input x

def Logistic_Regression(x,y,alpha,m,c):
  hp = sigmoid(m*x+c)         
  dL_dm = -(y-hp)*x                   
  dL_dc = -(y-hp)
  m = m - alpha*(dL_dm) 
  c = c - alpha*(dL_dc) 
  return m,c

#we run the loop for specific number of epochs to get the desired result
#In each epochs the logistic function is applied on every training example and the weights are updated

for i in range (epochs):
  for j in range (100):
    m,c = Logistic_Regression(X[j],Y[j],alpha,m,c)
  if(i%2000==0):                                                                      #we plot the graph on specific iterations only 
    plt.scatter(data['SepalWidthCm'][:50],data['Species'][:50],color="blue")              #plotting the data points
    plt.scatter(data['SepalWidthCm'][50:100],data['Species'][50:100],color="red")
    x_plot = data['SepalWidthCm']
    y_plot = sigmoid((x_plot)*m+c)                                #predicting y from current m and c values
    plt.plot(x_plot,y_plot,color='green')                   #plotting regression line
    plt.show()

#Print the weights(the variables we initialiized randolmly)
print(m,c)
#we are using the logistic regression function available in the sklearn libraray for comparison 

reg = LogisticRegression()
reg.fit(X[:100],Y[:100])

print(reg.coef_[0][0])
print(reg.intercept_[0])

m_lib=reg.coef_[0][0]
c_lib=reg.intercept_[0]

#we plot the graph using the weights produced by the logistic regression function 

plt.scatter(data['SepalWidthCm'][:50],data['Species'][:50],color="blue")              #plotting the data points
plt.scatter(data['SepalWidthCm'][50:100],data['Species'][50:100],color="red")
x_plot = data['SepalWidthCm'][:100]
y_plot = sigmoid((x_plot)*m_lib+c_lib)                                #predicting y from current m and c values
plt.plot(x_plot,y_plot,color='green')                   #plotting regression curve
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10736196319018405, 'fraction_numerical': 0.026352481873954267, 'mean_word_length': 3.430512662137122, 'pattern_counts': {'":': 3, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '160740', 'license': 'MIT', 'n_tokens_mistral': 2409, 'n_tokens_neox': 2227, 'n_words': 721, 'path': 'Logistic_Regression.ipynb', 'repo_name': 'm607stars/Machine-Learning-Algorithms', 'size': '7172'}