<jupyter_start><jupyter_text>This notebook is on Logistic Regression which is used for Classification of the Target Variable into discrete CategoriesWe will supplement the content of this notebook with the accompanying powerpoint presentation on Logistic Regression<jupyter_code>from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

import math

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn import linear_model
import scipy.stats as stats<jupyter_output><empty_output><jupyter_text>Data set contains the Age and Whether the respondent bought insurance or not.Notice, the Target Variable is NOT on a "Continuous" spectrum of values. Itis in two "Discrete" categories "Yes" and "No"<jupyter_code>has_insurance = pd.read_csv('../data/Insurance.csv')
has_insurance.head(25)<jupyter_output><empty_output><jupyter_text>As usual, let's try to do a scatter plot of the data<jupyter_code>plt.scatter(has_insurance['Age'], has_insurance['Has_Insurance'], color='red')
plt.title('Age Vs Has Insurance', fontsize=14)
plt.xlabel('Age', fontsize=14)
plt.ylabel('Has_Insurance', fontsize=14)
plt.grid(True)
plt.show()<jupyter_output><empty_output><jupyter_text>Does not look that great!! Not like what we have seen in Linear Regression. But let's get the Coefficients and Intercept and try to fit a straight line as we did in Linear Regression<jupyter_code>x = has_insurance[['Age']]
y = has_insurance['Has_Insurance']

x_list = has_insurance['Age']
# with sklearn
regr = linear_model.LinearRegression()
regr.fit(x, y)

print('Intercept: \n', regr.intercept_)
print('Coefficients: \n', regr.coef_)
def get_y_pred(x, slope, intercept):
    return (x * slope + intercept)

response_value_list = []
for x_i in x_list:
    response_value_list.append((get_y_pred(x_i, regr.coef_, regr.intercept_)))

plt.scatter(has_insurance['Age'], has_insurance['Has_Insurance'], color='red')
plt.plot(x_list, response_value_list, color = "g")
plt.title('Age Vs Has Insurance', fontsize=14)
plt.xlabel('Age', fontsize=14)
plt.ylabel('Has_Insurance', fontsize=14)
plt.grid(True)
plt.show()<jupyter_output><empty_output><jupyter_text>Now we will apply our Sigmoid Function (refer powerpoint presentation) and find the Best Fit line for Logistic Regression<jupyter_code>def sigmoid(x):
    i = 0
    a = []
    while i < len(x):
        pred = regr.coef_ * x[i] + regr.intercept_
        b = 1/(1+math.exp(-pred))
        if b < 0.6:
            b = 0
        else:
            b = 1
        a.append(b)
        i += 1
    return a

s = sigmoid(sorted(x_list))
plt.scatter(has_insurance['Age'], has_insurance['Has_Insurance'], color='red')
plt.plot(sorted(x_list), s, color="g")
plt.show<jupyter_output><empty_output><jupyter_text>We will split our data to two sets "Train" (to train the model) and "Test" (to test the prediction of the model. We have arbitrarily decided  for a 80 - 20 split<jupyter_code>X = has_insurance[['Age']]
y = has_insurance['Has_Insurance']
from sklearn.model_selection import train_test_split  
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
X_train
X_test<jupyter_output><empty_output><jupyter_text>This time we get a Logistic Regression model (instead of a LinearRegression) model from scikitlearn.After this we **"Train"** the model with our "Train" data, Predict using the Model and **"Test"** the prediction against the "Test" data and get a **"Score"** of how the model performed.<jupyter_code>regr = linear_model.LogisticRegression()
regr.fit(X_train, y_train)
regr.predict(X_test)
regr.score(X_test, y_test)<jupyter_output>C:\Users\jacob\Anaconda3\lib\site-packages\sklearn\linear_model\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
  FutureWarning)<jupyter_text>we also got the prediction for a specific Age<jupyter_code>regr.predict([[28]])
regr.predict([[29]])
regr.predict([[55]])<jupyter_output><empty_output><jupyter_text>We got the Probability of "No (will not buy)" and "Yes (will buy)" for the responents in our Test data set<jupyter_code>regr.predict_proba(X_test)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1116642958748222, 'fraction_numerical': 0.010194404931247037, 'mean_word_length': 5.905073649754501, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 36, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '335161', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1511, 'n_tokens_neox': 1437, 'n_words': 399, 'path': 'ExampleNotebooks/Logistic Regression.ipynb', 'repo_name': 'TimDuffy12/ml-app', 'size': '4218'}