<jupyter_start><jupyter_text>Logistic Regression  DataAn experiment was conducted on 5000 participants to study the effects of age and physical health on hearing loss, specifically the ability to hear high pitched tones. This data displays the result of the study in which participants were evaluated and scored for physical ability and then had to take an audio test (pass/no pass) which evaluated their ability to hear high frequencies. The age of the user was also noted. Is it possible to build a model that would predict someone's liklihood to hear the high frequency sound based solely on their features (age and physical score)?* Features    * age - Age of participant in years    * physical_score - Score achieved during physical exam* Label/Target    * test_result - 0 if no pass, 1 if test passed<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
df = pd.read_csv('../Data/hearing_test.csv')
df.head()<jupyter_output><empty_output><jupyter_text>1) Exploratory Data Analysis and Visualization<jupyter_code>df.info()
df.describe().transpose()<jupyter_output><empty_output><jupyter_text>check how many people passed or not passed the test<jupyter_code>df['test_result'].value_counts()
# visualize the data
sns.countplot(data=df, x='test_result');<jupyter_output><empty_output><jupyter_text>------ check the distribution of features vs label<jupyter_code>plt.figure(dpi=150)
sns.boxplot(data=df, x='test_result', y='age');<jupyter_output><empty_output><jupyter_text>As we can see from boxplot, age of people who didn't pass the test tends to be older which makes sense.However there are some outliers such under 30 years old and still didn't pass the test.On the flip side, younger people tends to perform better and there are some outliers who are older than 60 yr and still pass the test.<jupyter_code>plt.figure(dpi=150)
sns.boxplot(data=df, x='test_result', y='physical_score');<jupyter_output><empty_output><jupyter_text>There is a sharp contrast between who pass the test and who didn't pass the test. The higher the physical score, you can expect to pass the test. ------- We can check the relationship between physical score and age.<jupyter_code>plt.figure(dpi=150)
sns.scatterplot(data=df, x='age', y='physical_score', hue='test_result', alpha=0.5);<jupyter_output><empty_output><jupyter_text>We can see from scatterplot, there are two group who passed the test and who didn't pass the test.- for people who passed the test are younger people with higher physical_score.- on the other hand, people who didn't pass the test tends to be older people with lower physical score. ----- As dataset is quite small, we can use pairplot to check the relationship between features and labels.<jupyter_code>sns.pairplot(df, hue='test_result');<jupyter_output><empty_output><jupyter_text>We can see the distribution of age, physical score. Plus age vs physical_score scatterplot. -------- We can use heatmap to see how correlated features are.<jupyter_code>sns.heatmap(data=df.corr(), annot=True);<jupyter_output><empty_output><jupyter_text>-----  When scatterplot doesn't informative enough for some features and labels Esentially if we find this kind of scatter plot, we might want to use box plot instead.Because based on the chart, we can see there are two classes and it is showing the distribution.<jupyter_code>sns.scatterplot(data=df, x='physical_score', y='test_result');
sns.scatterplot(x='age',y='test_result',data=df)<jupyter_output><empty_output><jupyter_text>---------  3D scatterplothttps://matplotlib.org/3.1.1/gallery/mplot3d/scatter3d.html<jupyter_code>from mpl_toolkits.mplot3d import Axes3D  # noqa: F401 unused import

import matplotlib.pyplot as plt
import numpy as np

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

ax.scatter(df['age'], df['physical_score'], df['test_result'], c=df['test_result']);

ax.set_xlabel('Age')
ax.set_ylabel('Physical Score')
ax.set_zlabel('Test Result')<jupyter_output><empty_output><jupyter_text>We can see that there are clearly two clusters who passed and who didn't pass the test. --------  2) Model Training  Train | Test Split and Scaling<jupyter_code># seperate featrues and labels
X = df.drop('test_result', axis=1)
y = df['test_result']
# split train, test
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=101)
# scaling
scaler = StandardScaler()

scaled_X_train = scaler.fit_transform(X_train)
scaled_X_test = scaler.transform(X_test)<jupyter_output><empty_output><jupyter_text>Logistic Regression Model<jupyter_code>from sklearn.linear_model import LogisticRegression
lr_model = LogisticRegression()

lr_model.fit(scaled_X_train, y_train)
y_predict = lr_model.predict(scaled_X_test)
y_predict
y_pred = lr_model.predict_proba(X_test) # using probability prediction
y_pred<jupyter_output><empty_output><jupyter_text>-------  Coefficient InterpretationBased on the result below:* We can expect the **odds** of passing the test to **decrease** (the original coeff was negative) per unit increase of the age.* We can expect the **odds** of passing the test to **increase** (the original coeff was positive) per unit increase of the physical score.* Based on the ratios with each other, the physical_score indicator is a stronger predictor than age.<jupyter_code>lr_model.coef_
df.head(2)<jupyter_output><empty_output><jupyter_text>----------  Model Performance on Classification Tasks<jupyter_code>from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
y_pred = lr_model.predict(scaled_X_test)<jupyter_output><empty_output><jupyter_text>Accuracy Score- (TP+PN / Total)<jupyter_code># calculating accuracy score 
accuracy_score(y_test, y_pred)<jupyter_output><empty_output><jupyter_text>Confusion Matrix<jupyter_code>confusion_matrix(y_test, y_pred)<jupyter_output><empty_output><jupyter_text>As the above confusion matrix returns the values, we can visualize them in better way.  Plotting Confusion Matrix<jupyter_code>from sklearn.metrics import plot_confusion_matrix
plot_confusion_matrix(lr_model, scaled_X_test, y_test);
# scaling is also possible
plot_confusion_matrix(lr_model, scaled_X_test, y_test, normalize='all');
plot_confusion_matrix(lr_model, scaled_X_test, y_test, normalize='true');<jupyter_output><empty_output><jupyter_text>Classification Report : Recall, Precision, F1-Score per class<jupyter_code>print(classification_report(y_test, y_pred))<jupyter_output>precision    recall  f1-score   support

           0       0.92      0.89      0.91       193
           1       0.93      0.95      0.94       307

    accuracy                           0.93       500
   macro avg       0.93      0.92      0.93       500
weighted avg       0.93      0.93      0.93       500<jupyter_text>From the above report, we can see that precision, reall, f1-score and support are calculated per class (in this case : 0 and 1).When we look at the number, `overall accuracy: 0.93` is not very far off from `each class precision and recall, f1-score`.Also `support` tells us whether the dataset is balance or not.   Overall Precision and Recall Score<jupyter_code>from sklearn.metrics import precision_score, recall_score
recall_score(y_test, y_pred)
precision_score(y_test, y_pred)<jupyter_output><empty_output><jupyter_text>--------  Evaluating Curves and AUC<jupyter_code>from sklearn.metrics import plot_precision_recall_curve, plot_roc_curve<jupyter_output><empty_output><jupyter_text>ROC Curve<jupyter_code>fig, ax = plt.subplots(figsize=(12,5), dpi=150)
plot_roc_curve(lr_model, scaled_X_test, y_test, ax=ax);<jupyter_output><empty_output><jupyter_text>Precision Recall Curve<jupyter_code>fig, ax = plt.subplots(figsize=(12,5), dpi=150)
plot_precision_recall_curve(lr_model, scaled_X_test, y_test, ax=ax);<jupyter_output><empty_output><jupyter_text>---------  using probablity direclty to see the classification<jupyter_code>lr_model.predict_proba(scaled_X_test)
lr_model.predict_proba(scaled_X_test)[0]
y_test[0]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09429797670141017, 'fraction_numerical': 0.015205395462906193, 'mean_word_length': 5.7853577371048255, 'pattern_counts': {'":': 0, '<': 92, '<?xml version=': 0, '>': 92, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '589533', 'license': 'MIT', 'n_tokens_mistral': 2692, 'n_tokens_neox': 2557, 'n_words': 781, 'path': 'Machine Learning & Data Science Masterclass - JP/11-Logistic-Regression-Models/00-Logistic-Regression.ipynb', 'repo_name': 'ptyadana/probability-and-statistics-for-business-and-data-science', 'size': '8155'}