<jupyter_start><jupyter_text>$$ \min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^T w + c)) + 1) . $$ $$\min_{w, c} \frac{1 - \rho}{2}w^T w + \rho \|w\|_1 + C \sum_{i=1}^n \log(\exp(- y_i (X_i^T w + c)) + 1)$$ Here C is $$\frac{1}{\lambda}$$ $$  J = \sum y^{i}*log(h(x^i)) + (1-y^{i})log(1-h(x^i))  $$<jupyter_code>from sklearn.datasets import load_iris
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score,recall_score,precision_score,f1_score,roc_auc_score
from sklearn.linear_model import LogisticRegression
from scipy.special import expit
%matplotlib inline
iris_data = load_iris()
X = iris_data.data
y = iris_data.target
iris_data.target_names<jupyter_output><empty_output><jupyter_text>Pick only two classes to start with lets pick setosa and versicolor<jupyter_code>X = X[:100,:]
y = y[:100]
f,ax = plt.subplots(1,1)
ax.scatter(X[:,0],y,c=y)
ax.set_xlabel(iris_data.feature_names[0])
ax.set_ylabel("Class")<jupyter_output><empty_output><jupyter_text>Odds ratio is Pa/(1-Pa)log of odds is logit$$log(P/(1-P)) = y $$$$ e^y = \frac{P}{1-P} $$$$ P = \frac{1}{1+e^{-y}} $$  Sigmoid Function<jupyter_code>x = np.linspace(-5,5,1000)
sigmoid_x = 1.0/(1.0 + np.exp(-x))
plt.plot(x,y)
expit?
Xtrain,Xtest,ytrain,ytest = train_test_split(X,y)
binary_classifier = LogisticRegression(C = 1e4)
binary_classifier.fit(Xtrain[:,0].reshape(-1,1),ytrain)
ypreds = binary_classifier.predict(Xtest[:,0].reshape(-1,1))
print(accuracy_score(ytest,ypreds))
print(recall_score(ytest,ypreds))
print(precision_score(ytest,ypreds))
print(f1_score(ytest,ypreds))
print(roc_auc_score(ytest,ypreds))
from sklearn.metrics import confusion_matrix
confusion_matrix(ypreds,ytest)
probs = binary_classifier.predict_proba(Xtest[:,0].reshape(-1,1))
probs.shape
f,ax = plt.subplots(1,1)
ax.plot(Xtest[:,0],probs[:,1],'o',label="Probabilities")
ax.plot(Xtest[:,0],ytest,'<',label="Actual")
ax.plot(Xtest[:,0],ypreds,'*',label="Predictions")
ax.legend(loc='best')
ax.axhline(0.5)
ax.axvline(5.37,color='red')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1925233644859813, 'fraction_numerical': 0.02850467289719626, 'mean_word_length': 7.36328125, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '232514', 'license': 'FTL', 'n_tokens_mistral': 945, 'n_tokens_neox': 930, 'n_words': 147, 'path': 'notebooks/LogisticRegression.ipynb', 'repo_name': 'Jayaprakash-16/Bessant', 'size': '2140'}