<jupyter_start><jupyter_text>Data Presentation Project: "Stroke Prediction Model" Student Name: <NAME> Student ID: BF21DSY004The below data is from: https://www.kaggle.com/fedesoriano/stroke-prediction-dataset ---  INTRODUCTIONAccording to the World Health Organization (WHO) stroke is the 2nd leading cause of death globally, responsible for approximately 11% of total deaths. This dataset is used to predict whether a patient is likely to get stroke based on the input parameters like gender, age, various diseases, and smoking status. Each row in the data provides relavant information about the patient.  In order for us to predict the stroke probability, there are 12 attributes that we need to take into account in this dataset:1) id: unique identifier,2) gender: "Male" or "Female"3) age: age of the patient,4) hypertension: 0 if the patient doesn't have hypertension, 1 if the patient has hypertension,5) heart_disease: 0 if the patient doesn't have any heart diseases, 1 if the patient has a heart disease,6) ever_married: "No" or "Yes",7) work_type: "children", "Govt_job", "Never_worked", "Private" or "Self-employed",8) Residence_type: "Rural" or "Urban",9) avg_glucose_level: average glucose level in blood,10) bmi: body mass index,11) smoking_status: "formerly smoked", "never smoked", "smokes" or "Unknown", The target column:12) stroke: 1 if the patient have a stroke or 0 if not*Note: "Unknown" in smoking_status means that the information is unavailable for this patient* ---  PROJECT OUTLINE 1) DATA PREPROCESSINGA) IMPORT LIBRARIES AND DATASETB) GENERAL DESCRIPTION AND DATA VISUALIZATIONC) DEALING WITH NULL VALUESD) DEALING WITH CATEGORICAL DATA AND DATA NORMALIZATION 2) FITTING AND TESTING MULTIPLE MODELA) SPLITTING THE DATASETB) FITTING THE MODEL 3) BUILDING THE BEST MODEL FOR PREDICTIONA) ANALYSIS OF 2 MODELS: CONFUSION MATRIX AND CLASSIFICATION REPORTB) FIND THE BEST MODEL:- XGBOOST- LOGISTIC REGRESSIONC) APPLY NEW PARAMETER AND TRAIN ITD) FIND THE BEST THRESHOLD 4) PREDICTION 5) CONCLUSIONS ---  1) DATA PREPROCESSING  1.A. IMPORTING LIBRARIES AND DATASET<jupyter_code># Importing necessary libraries
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
# Importing the dataset
stroke_df = pd.read_csv('healthcare-dataset-stroke-data.csv')
# Displaying the dataframe
stroke_df.head(10)<jupyter_output><empty_output><jupyter_text>1.B. GENERAL DESCRIPTION AND DATA VISUALIZATION<jupyter_code># Checking the number of rows and columns in the dataframe
stroke_df.shape
# Checking if there are any missing values
stroke_df.info()
# Describing the dataset's statistical properties
stroke_df.describe().round(decimals=2)
# Here we check the distribution of stroke and no stroke in target column
stroke_df['stroke'].value_counts()
# Visualizing target distribution using pie chart
stroke_df['stroke'].value_counts().plot.pie()

print('No Stroke', round(stroke_df['stroke'].value_counts()[0]/len(stroke_df) * 100,2), '% of the dataset')
print('Stroke', round(stroke_df['stroke'].value_counts()[1]/len(stroke_df) * 100,2), '% of the dataset')
# Visualizing target distribution using bar chart
colors = ["#0101DF", "#DF0101"]

sns.countplot('stroke', data=stroke_df, palette=colors)
plt.title('Target Distributions \n (0: No Stroke || 1: Stroke)', fontsize=14)
# Visualizing Histogram of each attribute
import matplotlib.pyplot as plt
stroke_df.hist(bins=50, figsize=(20, 15))
plt.show()
# Visualizing the data correlation matrix
cor_matrix = stroke_df.corr().round(2)

sns.heatmap(cor_matrix, annot = True)
# Dropping irrelevant data
stroke_df.drop('id', axis = 1, inplace = True)

cor_matrix = stroke_df.corr().round(2)

sns.heatmap(cor_matrix, annot = True)
# Setting up target variable in y and drop the target column from data frame
y = stroke_df['stroke']
stroke_df.drop('stroke', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>1.C. DEALING WITH NULL VALUES Since the only column that has null values is BMI, then we only have to deal with that column only. Because the column 'BMI' is a numerical column, we can choose either to fill NaN with mean or median of the data. For this particular project, I am going to change the null values with median (since the data is skewed).<jupyter_code># Substitute NaN with median values
stroke_df['bmi'].fillna(stroke_df['bmi'].median(), inplace = True)
# Check if there are any NaN values
stroke_df['bmi'].isnull().any()<jupyter_output><empty_output><jupyter_text>1.D. DEALING WITH CATEGORICAL DATA AND DATA NORMALIZATION<jupyter_code># Change the categorical attributes which only has 2 values into numeric attributes
# for gender
gender = {'Male': 1,
          'Female': 0}

stroke_df['gender'] = stroke_df['gender'].replace(gender)

# for ever_married
marital_status = {'Yes': 1,
                  'No': 0}

stroke_df['ever_married'] = stroke_df['ever_married'].replace(marital_status)

# for Residence_type
residence = {'Urban': 1,
            'Rural': 0}

stroke_df['Residence_type'] = stroke_df['Residence_type'].replace(residence)
stroke_df.head()<jupyter_output><empty_output><jupyter_text>As we can see now, these values have changed:for gender: 1 = Male, 0 = Femalefor ever_married: 1 = Yes, 0 = Nofor Residence_type: 1 = Urban, 0 = Rural<jupyter_code># Import required data transformation and set up column transformer object
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
from sklearn.preprocessing import StandardScaler

ct = ColumnTransformer([('cat', OneHotEncoder(),[5,9])], remainder = "passthrough")
# The transformed data is now saved in a new data frame
x = ct.fit_transform(stroke_df)
# Normalizing the whole dataframe
sc = StandardScaler()

x = sc.fit_transform(x)
x.shape
stroke_df.shape<jupyter_output><empty_output><jupyter_text>Notice that our data matrix shape has changed, this is due to the onehotencoder increasing the number of columns based on the categorical data types. Explanation:Since there are two columns of categorical data types, 'work_type' and 'smoking_status' where in total, both columns have 9 unique values, making the total columns change from 10 to 17 (10-2+9=17). ---  2) FITTING AND TESTING MULTIPLE MODELS  2.A. SPLITTING DATASET BEFORE TESTING In this project, we are splitting the dataset into train and test sets with ratio of 75 : 25.<jupyter_code>from sklearn import model_selection

x_train, x_test, y_train, y_test = model_selection.train_test_split(x, y, test_size=0.25, random_state=5)<jupyter_output><empty_output><jupyter_text>2.B. FITTING THE MODEL<jupyter_code># First, we import the classifiers that we want to use
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors  import KNeighborsClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from lightgbm import LGBMClassifier
from xgboost import XGBClassifier
# Then, we start appending the classifier into models list
results=[]
names=[]

models = []
models.append(('LR',LogisticRegression(solver='liblinear',multi_class='ovr')))
models.append(('CARD',DecisionTreeClassifier()))
models.append(('KNN',KNeighborsClassifier()))
models.append(('LDA',LinearDiscriminantAnalysis()))
models.append(('LGBM',LGBMClassifier()))
models.append(('XGB',XGBClassifier(use_label_encoder=False)))
#Evaluate each model in turn
for name, model in models:
    kfold = model_selection.KFold(n_splits=10,random_state=7, shuffle = True)
    cv_results = model_selection.cross_val_score(model,x_train,y_train,cv = kfold, scoring='accuracy')
    results.append(cv_results)
    names.append(name)
    msg="%s:%f(%f)"%(name, cv_results.mean(), cv_results.std())
    print(msg)
# Using boxplot algorithm comparison
fig = plt.figure()
fig.suptitle('Algorithm Comparison')
ax = fig.add_subplot(111)
plt.boxplot(results)
ax.set_xticklabels(names)
plt.show()
# Now, we visualize the comparison using ROC curve
from sklearn.metrics import roc_curve
from sklearn.model_selection import cross_val_predict
from sklearn.model_selection import cross_val_score

# Logistic Regression
lr = LogisticRegression()
lr_results = cross_val_score(lr, x, y, scoring = 'recall')
print (lr_results.mean(), '+/-',lr_results.std())

y_score_lr = cross_val_predict(lr, x, y, cv = 10, method = 'decision_function')
fpr_lr, tpr_lr, threshold_lr = roc_curve(y, y_score_lr)
# Decission Tree Classifier
dtc = DecisionTreeClassifier()
dtc_results = cross_val_score(dtc,x, y, scoring = 'recall')
print (dtc_results.mean(), '+/-',dtc_results.std())

y_score_dtc = cross_val_predict(dtc,x, y, cv = 10, method = 'predict_proba')
fpr_dtc, tpr_dtc, threshold_dtc = roc_curve(y, y_score_dtc[:,1]) 
# K Neighbor Classifier
knn = KNeighborsClassifier()
knn_results = cross_val_score(knn, x, y, scoring = 'recall')
print (knn_results.mean(), '+/-',knn_results.std())

y_score_knn = cross_val_predict(knn, x, y, cv = 10, method = 'predict_proba')
fpr_knn, tpr_knn, threshold_knn = roc_curve(y, y_score_knn[:,1])
# Linear Discriminant Analysis
lda = LinearDiscriminantAnalysis()
lda_results = cross_val_score(lda, x, y, scoring = 'recall')
print (lda_results.mean(), '+/-',lda_results.std())

y_score_lda = cross_val_predict(lda, x, y, cv = 10, method = 'predict_proba')
fpr_lda, tpr_lda, threshold_lda = roc_curve(y, y_score_lda[:,1])
# LightGBM
lgbm = LGBMClassifier()
lgbm_results = cross_val_score(lgbm, x, y, scoring = 'recall')
print (lgbm_results.mean(), '+/-',lgbm_results.std())

y_score_lgbm = cross_val_predict(lgbm, x, y, cv = 10 ,method = 'predict_proba')
fpr_lgbm, tpr_lgbm, threshold_lgbm = roc_curve(y, y_score_lgbm[:,1])
# XGBoost
xgb = XGBClassifier(use_label_encoder=False)
xgb_results = cross_val_score(xgb, x, y, scoring = 'recall')
print (xgb_results.mean(), '+/-',xgb_results.std())

y_score_xgb = cross_val_predict(xgb, x, y, cv = 10 ,method = 'predict_proba')
fpr_xgb, tpr_xgb, threshold_xgb = roc_curve(y, y_score_xgb[:,1])
# ROC Curve for the models above
plt.plot(fpr_lr, tpr_lr, label='Logistic Regression')
plt.plot(fpr_dtc, tpr_dtc,label='CARD')
plt.plot(fpr_knn, tpr_knn,label='KNN')
plt.plot(fpr_lda, tpr_lda,label='LDA')
plt.plot(fpr_lgbm, tpr_lgbm,label='LGBM')
plt.plot(fpr_xgb, tpr_xgb,label='XGB')
plt.legend()
plt.plot([0, 1], [0, 1], 'k--')
plt.axis([0, 1, 0, 1])
plt.xlabel('False Positive Rate', fontsize=16)
plt.ylabel('True Positive Rate', fontsize=16)
plt.figure(figsize=(10, 6))
plt.show()
plt.show
from sklearn.metrics import roc_auc_score
print ('AUC of Logistic Regression:',roc_auc_score(y, y_score_lr))
print ('AUC of Decision Tree:',roc_auc_score(y, y_score_dtc[:,1]))
print ('AUC of KNeighborsClassifier:',roc_auc_score(y, y_score_knn[:,1]))
print ('AUC of Linear Discriminant Analysis:',roc_auc_score(y, y_score_lda[:,1]))
print ('AUC of LightGBM:',roc_auc_score(y, y_score_lgbm[:,1]))
print ('AUC of XGBoost:',roc_auc_score(y, y_score_xgb[:,1]))<jupyter_output>AUC of Logistic Regression: 0.8391508845503388
AUC of Decision Tree: 0.5511583466141877
AUC of KNeighborsClassifier: 0.6020010922108512
AUC of Linear Discriminant Analysis: 0.8341624056398399
AUC of LightGBM: 0.8323844648290757
AUC of XGBoost: 0.8450535323767813<jupyter_text>From observation, the two highest performing classifier are XGBoost and Logistic Regression with 84.50% and 83.93%. ---  3) BUILDING THE BEST MODEL FOR PREDICTION  3.A. ANALYSIS OF 2 MODELS: CONFUSION MATRIX AND CLASSIFICATION REPORT  CONFUSION MATRIX<jupyter_code># XGBoost
xgb.fit(x_train,y_train)
predictions1 = xgb.predict(x_test)
cm1 = confusion_matrix(y_test,predictions1)
fig, ax= plt.subplots(figsize=(6,3))
sns.heatmap(cm1, annot=True, ax = ax,linewidths=.5); #annot=True to annotate cells

# Labels, title and ticks
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels'); 
ax.set_title('Confusion Matrix'); 
ax.xaxis.set_ticklabels(['Positive', 'Negative']); ax.yaxis.set_ticklabels(['Negative', 'Positive']);

# TP, FP, TN, and FN count
tn, fp, fn, tp = confusion_matrix(y_test, predictions1).ravel()
print('True Positive:', tp,';',
      'False Positive:', fp,';',
      'True Negative:', tn,';',
      'False Negative', fn)
# Logistic Regression
lr.fit(x_train,y_train)
predictions2 = lr.predict(x_test)
cm2 = confusion_matrix(y_test,predictions2)
fig, ax= plt.subplots(figsize=(6,3))
sns.heatmap(cm2, annot=True, ax = ax,linewidths=.5); #annot=True to annotate cells

# Labels, title and ticks
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels'); 
ax.set_title('Confusion Matrix'); 
ax.xaxis.set_ticklabels(['Positive', 'Negative']); ax.yaxis.set_ticklabels(['Negative', 'Positive']);

# TP, FP, TN, and FN count
tn, fp, fn, tp = confusion_matrix(y_test, predictions2).ravel()
print('True Positive:', tp,';',
      'False Positive:', fp,';',
      'True Negative:', tn,';',
      'False Negative', fn)<jupyter_output>True Positive: 0 ; False Positive: 0 ; True Negative: 1213 ; False Negative 65<jupyter_text>In this project, maximizing recall is prioritized since we want to eliminate as much as false negative as possible. [False Negative: A person is diagnosed to not have stroke, when they are actually likely to have one.]  CLASSIFICATION REPORT<jupyter_code># XGBoost
print(classification_report(y_test,predictions1))
# Logistic Regression
print(classification_report(y_test,predictions2))<jupyter_output>precision    recall  f1-score   support

           0       0.95      1.00      0.97      1213
           1       0.00      0.00      0.00        65

    accuracy                           0.95      1278
   macro avg       0.47      0.50      0.49      1278
weighted avg       0.90      0.95      0.92      1278<jupyter_text>3.B. FINDING THE BEST MODEL  XGBoost<jupyter_code>from sklearn.model_selection import GridSearchCV

param_grid1 = {'solver': ['svd', 'lsqr', 'eigen']}

grid_search1 = GridSearchCV(XGBClassifier(), param_grid1, cv = 10, scoring='accuracy')

grid_search1.fit(x_train, y_train)

print("The best score for this model is",grid_search1.best_score_)<jupyter_output>The best score for this model is 0.9517229656222803<jupyter_text>Logistic Regression<jupyter_code># Penalty Testing to avoid overfitting
# Using RIDGE REGRESSION (L2 PENALTY)
from sklearn.linear_model import Ridge

ridge = Ridge()

parameters= {'alpha':[2,4,6,8,10]}

ridge_reg=GridSearchCV(ridge, parameters,scoring='neg_mean_squared_error',cv=10)
ridge_reg.fit(x_train,y_train)
print("The best parameter is:",ridge_reg.best_params_)
print("The best score for this penalty is",ridge_reg.best_score_)
#Using LASSO REGRESSION (L1 PENALTY)
from sklearn.linear_model import Lasso

lasso = Lasso()

parameters= {'alpha':[2,4,6,8,10]}

lasso_reg=GridSearchCV(lasso, param_grid=parameters, scoring='neg_mean_squared_error', cv=10)
lasso_reg.fit(x_train,y_train)
print("The best parameter is:",lasso_reg.best_params_)
print('The best score for this penalty is: ',lasso_reg.best_score_)<jupyter_output>The best parameter is: {'alpha': 2}
The best score for this penalty is:  -0.04573885039181573<jupyter_text>The above testing points out that the L2 penalty with alpha = 10 would be more effective for our model (since -0.046<-0.042)<jupyter_code>param_grid2 = {
    'penalty':['l2'],
    'C':[1.0,10.0,50.0],
    'solver':['newton-cg', 'liblinear']
}

model_lr= LogisticRegression()

grid_search2 = GridSearchCV(
    model_lr, param_grid2, cv=10, scoring='accuracy')
grid_search2.fit(x_train, y_train)
print("The best score for this model is",grid_search2.best_score_)<jupyter_output>The best score for this model is 0.9519840622280246<jupyter_text>Since both models with optimized parameter have a similar best accuracy score, we can train either one of them for our prediction.  3.C. APPLYING NEW PARAMETER AND TRAINING THE DATASET For this particular project, I am choosing XGBoost to train my dataset<jupyter_code># Create an instance of the algorithm using parameters from best_estimator_ property
xgb = grid_search1.best_estimator_
# Use the whole dataset to train the model
X = np.append(x_train, x_test, axis=0)
Y = np.append(y_train, y_test, axis=0)
# Train the model
xgb.fit(X, Y)<jupyter_output><empty_output><jupyter_text>3.D. FINDING THE BEST THRESHOLD<jupyter_code># Finding Best Threshold using G-mean
gmeans = np.sqrt(tpr_xgb * (1-fpr_xgb))

# Locate the index of the largest g-mean
ix = np.argmax(gmeans)
print('Best Threshold=%f, G-Mean=%.3f' % (threshold_xgb[ix], gmeans[ix]))
# Finding the best threshold using Youden's J Statistic
J = tpr_xgb - fpr_xgb
ix = np.argmax(J)
best_thresh = threshold_xgb[ix]
print('Best Threshold=%f' % (best_thresh))
# Plotting the roc curve with the threshold
from matplotlib import pyplot
pyplot.plot([0,1], [0,1], linestyle='--', label='No Skill')
pyplot.plot(fpr_xgb, tpr_xgb, marker='.', label='Logistic')
pyplot.scatter(fpr_xgb[ix], tpr_xgb[ix], marker='o', color='black', label='Best')

# Axis labels
pyplot.xlabel('False Positive Rate')
pyplot.ylabel('True Positive Rate')
pyplot.legend()
pyplot.show()<jupyter_output><empty_output><jupyter_text>We take average of both best thrsholds that we can find (Using g-mean and J statistic) we found it to be 0.0505985. Therefore, if the probability belongs to '1' class is larger than 0.0505, then a person is predicted to have a stroke. ---  4) PREDICTION<jupyter_code># Define a function to test our prediction:
def prediction(data):
    data_scaled = sc.transform(ct.transform(data))
    predict = xgb.predict_proba(data_scaled)[:,1]
    if predict > 0.05:
        print("The person is likely to have a stroke.")
    else:
        print("The person is not likely have a stroke.")
# We create a new person's infomation to test our prediction
patient = pd.DataFrame([[0, 25, 0, 0, 0, 'Private', 1, 125, 25, 'Unknown']])

# We use our function to predict:
prediction(patient)<jupyter_output>The person is not likely have a stroke.
{'fraction_non_alphanumeric': 0.11368549415220885, 'fraction_numerical': 0.027936367163682724, 'mean_word_length': 5.899426386233269, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '154880', 'license': 'MIT', 'n_tokens_mistral': 6614, 'n_tokens_neox': 6117, 'n_words': 1791, 'path': 'stroke_classification_notebook.ipynb', 'repo_name': 'deknared/Stroke-Classification-Project', 'size': '18058'}