<jupyter_start><jupyter_text>**First, create a dataframe of *cleaned_data.csv*. Then, use scikit-learn's `train_test_split` to make a new set of training and testing data. This time, instead of `EDUCATION`, use `LIMIT_BAL`: the account's credit limit.**<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib as mpl
%matplotlib inline

plt.style.use('dark_background')
mpl.rcParams['figure.dpi'] = 200
mpl.rcParams['font.size'] = 3
data = pd.read_csv('cleaned_data.csv')
data.head()<jupyter_output><empty_output><jupyter_text>**Exploring the Response Variable and Concluding the Initial Exploration**<jupyter_code>data['default payment next month'].mean()
data.groupby('default payment next month')['ID'].count()<jupyter_output><empty_output><jupyter_text>_________________________________________________________________________**Next, train a logistic regression model using the training data from your split.**<jupyter_code>from sklearn.linear_model import LogisticRegression
lr = LogisticRegression()
lr
lr_model = LogisticRegression(C = 1.0, 
                              class_weight = None, 
                              dual = False, 
                              fit_intercept = True,
                             intercept_scaling = 1,
                             max_iter = 100,
                             multi_class = 'auto',
                             n_jobs = None,
                             penalty = 'l2',
                             random_state = None,
                             solver = 'warn',
                             tol = 0.0001,
                             verbose = 0,
                             warm_start = False)
lr_model
lr_model.C = 0.1
lr_model.solver = 'liblinear'
lr_model
X = data['EDUCATION'][0:10].values.reshape(-1,1)
X
y = data['default payment next month'][0:10].values
y
lr_model.fit(X, y)<jupyter_output><empty_output><jupyter_text>_________________________________________________________________________**Then, create the array of predicted probabilities for the testing data.**<jupyter_code>new_X = data['EDUCATION'][10:20].values.reshape(-1,1)
new_X
lr_model.predict(new_X)
data['default payment next month'][10:20].values
# Generating Synthetic Data
np.random.seed(seed = 1)

X = np.random.uniform(low = 0.0, high = 10.0, size = (1000,))
X[0:10]
np.random.seed(seed = 1)

slope = 0.25
intercept = -1.25

y = slope * X + np.random.normal(loc = 0.0, scale = 1, size = (1000,)) + intercept
plt.scatter(X, y, s = 1)
plt.show()
from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg
lin_reg.fit(X.reshape(-1,1), y)
print(f"Intercept:{lin_reg.intercept_}, Coefficient:{lin_reg.coef_}")
y_pred = lin_reg.predict(X.reshape(-1,1))
plt.scatter(X, y, s = 1)
plt.plot(X, y_pred, 'r')
plt.show()<jupyter_output><empty_output><jupyter_text>**Splitting the Data: Training and Testing Sets**<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(data['EDUCATION'].values.reshape(-1,1), data['default payment next month'].values, test_size = 0.2, random_state = 24)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)
print(np.mean(y_train), np.mean(y_test))<jupyter_output>0.223102526838873 0.21657603600225014<jupyter_text>**Note:** If the positive class is very rare in our dataset to evenly split the data we can pass stratify argument in the train_test_split function, which will ensure that the train and test data gets the same fraction of data for each class.<jupyter_code>example_lr = LogisticRegression(C=0.1, class_weight=None, dual=False, fit_intercept=True,
                                intercept_scaling=1, max_iter=100, multi_class='auto', n_jobs=None,
                                penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
                                verbose=0, warm_start=False)
example_lr.fit(X_train, y_train)
y_pred = example_lr.predict(X_test)
y_pred
is_correct = y_pred == y_test
np.mean(is_correct)
print(example_lr.score(X_test, y_test))
from sklearn.metrics import accuracy_score
print(accuracy_score(y_test, y_pred))
P = sum(y_test)
P
TP = sum( (y_test==1) & (y_pred==1) ) # Calculating True Positives
print(TP)

TPR = TP/P # Calculating True Positive Rate
print(TPR)
FN = sum((y_test == 1) & (y_pred == 0))
print(FN)

FNR = FN/P
print(FNR)
N = sum(y_test==0)
TN = sum((y_test==0) & (y_pred==0))
FP = sum((y_test==0) & (y_pred==1))
TNR = TN/N
FPR = FP/N
print('The true negative rate is {} and the false positive rate is {}'.format(TNR, FPR))
from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_test, y_pred))<jupyter_output>[[4178    0]
 [1155    0]]<jupyter_text>Obtaining Predicted Probabilities from a Trained Logistic Regression Model<jupyter_code>y_pred_proba = example_lr.predict_proba(X_test)
y_pred_proba
prob_sum = np.sum(y_pred_proba, 1)
prob_sum
prob_sum.shape
np.unique(prob_sum)
pos_proba = y_pred_proba[:, 1]
pos_proba
plt.hist(pos_proba)
plt.show()
mpl.rcParams['font.size'] = 12
plt.hist(pos_proba)
plt.xlabel('Predicted probability of positive class for testing data')
plt.ylabel('Number of samples')
plt.show()
pos_sample_pos_proba = pos_proba[y_test==1]
neg_sample_pos_proba = pos_proba[y_test==0]
plt.hist([pos_sample_pos_proba, neg_sample_pos_proba], histtype='barstacked')
plt.legend(['Positive samples', 'Negative samples'])
plt.xlabel('Predicted probability of positive class')
plt.ylabel('Number of samples')
plt.show()<jupyter_output><empty_output><jupyter_text>_________________________________________________________________________**Next, calculate the ROC AUC using the predicted probabilities and the true labels of the testing data. Compare this to the ROC AUC from using the `EDUCATION` feature.**<jupyter_code>from sklearn import metrics
fpr, tpr, thresholds = metrics.roc_curve(y_test, pos_proba)
print(fpr, tpr, thresholds)<jupyter_output>[0.         0.01842987 0.16419339 0.63140258 1.        ] [0.         0.004329   0.21212121 0.7030303  1.        ] [1.2549944  0.2549944  0.24007604 0.22576598 0.21207085]<jupyter_text>_________________________________________________________________________**Now, plot the ROC curve.**<jupyter_code>plt.plot(fpr, tpr, '*-')
plt.plot([0,1], [0,1], 'r--')
plt.legend(['Logistic Regression', "Random Chance"])
plt.xlabel('fpr (False Positive Rate)')
plt.ylabel('tpr (True Positive Rate)')
plt.title('ROC Curve (Receiver Operating Characteristic Curve)')
plt.show()<jupyter_output><empty_output><jupyter_text>**Note:**  In general, the ROC AUC is going to be between 0.5 and 1.The ROC AUC may also be referred to as the C-statistic.<jupyter_code>metrics.roc_auc_score(y_test, pos_proba)<jupyter_output><empty_output><jupyter_text>_________________________________________________________________________**Then, calculate the data for the precision-recall curve on the testing data using scikit-learn functionality.**<jupyter_code>precision, recall, thresholds = metrics.precision_recall_curve(y_test, pos_proba)
print(precision, recall, thresholds)<jupyter_output>[0.21657604 0.23536232 0.26315789 0.06097561 1.        ] [1.         0.7030303  0.21212121 0.004329   0.        ] [0.21207085 0.22576598 0.24007604 0.2549944 ]<jupyter_text>_________________________________________________________________________**Plot the precision-recall curve using matplotlib.**<jupyter_code>plt.plot(recall, precision, '*-')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision Recall Curve')
plt.show()<jupyter_output><empty_output><jupyter_text>_________________________________________________________________________**Now, use scikit-learn to calculate the area under the precision-recall curve.**<jupyter_code>metrics.auc(recall, precision) # auc is a function used to calculate area under curve<jupyter_output><empty_output><jupyter_text>_________________________________________________________________________**Finally, recalculate the ROC AUC, except this time do it for the training data.**<jupyter_code># getting the training samples probability predictions
y_train_prob = example_lr.predict_proba(X_train)

# retriving all positiv class probabilities
train_pos_proba = y_train_prob[:,1]

# getting the required parameters
fpr, tpr, thresholds = metrics.roc_curve(y_train, train_pos_proba)

# plotting roc-auc cuve
plt.plot(fpr, tpr, '*-')
plt.plot([0,1], [0,1], 'r--')
plt.legend(['Logistic Regression', "Random Chance"])
plt.xlabel('fpr (False Positive Rate)')
plt.ylabel('tpr (True Positive Rate)')
plt.title('ROC Curve for Training Data')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11664739884393063, 'fraction_numerical': 0.04115606936416185, 'mean_word_length': 4.778891115564463, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '188681', 'license': 'MIT', 'n_tokens_mistral': 3116, 'n_tokens_neox': 2796, 'n_words': 661, 'path': 'Internship mini project 1 to 7/Task#3 - Performing_Logistic_Regression_Preet_Mehta .ipynb', 'repo_name': 'ynpreet/Technocolabs-data-science-internship', 'size': '8650'}