<jupyter_start><jupyter_text>Logistic Regression Misal regresi linear yang digunakan untuk memprediksi suatu nilai variabel dependen memiliki rumus $$ y = \theta_{0} + \theta_{1} x_1 + \theta_{2} x_2 + ... + \theta_{n} x_n  $$ Maka rumus regresi logistik yang digunakan untuk mengategorikan secara biner dua variabel adalah$$ Ln(\frac{P}{1-P}) = \theta_{0} + \theta_{1} x_1 + \theta_{2} x_2 + ... + \theta_{n} x_n  $$ atau dapat dituliskan $$ P =  \frac{e^{\theta_{0} + \theta_{1} x_1 + \theta_{2} x_2 + ... + \theta_{n} x_n}}{1+e^{\theta_{0} + \theta_{1} x_1 + \theta_{2} x_2 + ... + \theta_{n} x_n}} $$ atau $$P =  \frac{1}{1+e^{-(\theta_{0} + \theta_{1} x_1 + \theta_{2} x_2 + ... + \theta_{n} x_n)}} $$$$P =  \frac{1}{1+e^{-\hat{y}}} $$ Secara mudah, kita dapat menggunakan gradient descent seperti pada rumus regresi linear. Untuk klasifikasi kategorinya$$ P = \{^{\ge0.5,class=1 }_{<0.5,class=0}  $$Sehingga, rumus konvergensinya dengan cost function Binary Cross Entropy adalah$$ \theta_j = \theta_j - \alpha \frac{1}{n} \frac{\partial }{\partial \theta_j} - \sum_{i=1}^{n} y_{i} ln(h_{\theta}(x_{i})) + (1-y_{i}) ln(1-h_{\theta}(x_{i}))  $$di mana$$ h_{\theta}(x_{i}) = \frac{1}{1+e^{-(\theta_{0} + \theta_{1} x_1 + \theta_{2} x_2 + ... + \theta_{n} x_n)}}  $$- $ y_{i} $ bernilai 1 atau 0 sesuai kelasnya- Sebenarnya, kita juga bisa menggunakan cost function MSE. Namun untuk klasifikasi biner, cost function BCE akan lebih akurat.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import random
from sklearn import datasets
from sklearn.model_selection import train_test_split
np.set_printoptions(precision=3, suppress=False)
iris = datasets.load_iris( as_frame=True)
df = iris.data
iris.target
df['is_setosa'] =0
for i in range(len(df)):
    if iris.target[i] == 0:
        df.loc[i, 'is_setosa'] = 1
    else:
        df.loc[i, 'is_setosa'] = 0
df
# 5 Data terakhir
# Bwt = Berat kucing
# Hwt = Berat hati kucing
df.tail()
df.describe()<jupyter_output><empty_output><jupyter_text>Analisis Logistic Regression mirip dengan Linear Regression, namun alih-alih menggunakan Fungsi Linear, kita menggunakan fungsi Sigmoid<jupyter_code>import torch
X = df.drop(['is_setosa'], axis=1).values

Y = df['is_setosa'].values
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.3, random_state=0)
class LogisticRegression(torch.nn.Module):
    def __init__(self, input_dim, output_dim):
        super(LogisticRegression, self).__init__()
        self.linear = torch.nn.Linear(input_dim, output_dim)
    
    def forward(self, x):
        outputs = torch.sigmoid(self.linear(x))
        return outputs
learningRate = 0.1
epochs = 1000

m = len(x_train) #Jumlah data
n = len(x_train[0]) #Jumlah parameter
X_tensor = torch.from_numpy(x_train.reshape( m, n)).float() #Membuat tensor dari data input
Y_tensor = torch.from_numpy(y_train.reshape( m, 1)).float() #Membuat tensor dari data output

model = LogisticRegression(n, 1)

if torch.cuda.is_available():
    model = model.cuda()
optimizer = torch.optim.SGD(model.parameters(), lr=learningRate)
criterion = torch.nn.BCELoss()
for epoch in range(epochs):
    if torch.cuda.is_available():
        inputs = X_tensor.cuda()
        targets = Y_tensor.cuda()
    else:
        inputs = X_tensor
        targets = Y_tensor
    
    optimizer.zero_grad()

    outputs = model(inputs)

    loss = criterion(outputs, targets)
    loss.backward()
    optimizer.step()

    if (epoch+1) % 25 == 0:
        print('Epoch [{}/{}], Loss: {:.4f}'.format(epoch+1, epochs, loss.item()))
PredictionProbability = model(torch.from_numpy(x_test).float().cuda()).cpu().detach().numpy()
plt.plot(x_test, y_test, 'ro')
plt.plot(x_test, PredictionProbability, 'bo')
plt.show()<jupyter_output><empty_output><jupyter_text>Confusion Matrix untuk klasifikasi Biner dapat dinyatakan dalam bentuk berikut:$$\begin{bmatrix}True\; Positive & False\; Negative \\False\; Positive & True\; Negative\end{bmatrix}$$Untuk confusion matrix dengan lebih dari dua klasifikasi, suatu model dikatakan bagus jika model tersebut membentuk confusion matrix yang mirip dengan matriks diagonal<jupyter_code>from sklearn.metrics import confusion_matrix
cf = confusion_matrix(y_test, PredictionProbability > 0.5, normalize='true')
sns.heatmap(cf, annot=True)
print(cf)<jupyter_output>[[1. 0.]
 [0. 1.]]
{'fraction_non_alphanumeric': 0.15305422100205904, 'fraction_numerical': 0.018531228551818806, 'mean_word_length': 4.485570890840653, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '530921', 'license': 'MIT', 'n_tokens_mistral': 1735, 'n_tokens_neox': 1664, 'n_words': 391, 'path': 'Logistic Regression/Logistic Regression Pytorch.ipynb', 'repo_name': 'karuniaperjuangan/Notebook-Machine-Learning', 'size': '4371'}