<jupyter_start><jupyter_text>There are no null values in the dataset. No categorical features and all are numerical( float and int)<jupyter_code># split data and label
M_Diabetes = Diabetes 
Outcome  = M_Diabetes['Outcome']
M_Diabetes.drop('Outcome', axis=1, inplace = True)
# Splitting data into positive and negative examples
Diabetes = pd.read_csv('Data/diabetes.csv')
Positives  = Diabetes[Diabetes['Outcome'] == 1]
Negatives = Diabetes[Diabetes['Outcome'] == 0]<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis<jupyter_code>Diabetes.sample(frac =0.1).head(n=5)
Diabetes.describe()
# From above, min value of Glucose, Blood Pressure, skin thickness, Insulin, BMI cannot be 0.

# Plotting distributions of features ----
# Visualize number of different classes 
ax = plt.rc("font", size = 10)
ax = Diabetes['Outcome'].value_counts().plot(kind='bar')
ax.set_title("Outcome distribution of patients \n")
ax.set_xlabel('Outcomes')
ax.set_ylabel('No. of patients')
for p in ax.patches:
    ax.annotate(format(p.get_height()),(p.get_x()+0.2,p.get_height()+0.2))
plt.savefig('Plots/pstients_distribution.png', bbox_inches = 'tight')    
plt.show()
# 500 people with diabetes and 268 without diabetes (inbalanced dataset)
# Hence, Accuracy is not a useful metric for machine learning algorithms.
# Other metrics should be used --> ROC_AUC, recall, precision, F1-score, sensitivity, specificity...
print(Diabetes.columns)<jupyter_output>Index(['Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness', 'Insulin',
       'BMI', 'DiabetesPedigreeFunction', 'Age', 'Outcome'],
      dtype='object')<jupyter_text>Data Cleaning<jupyter_code>Diabetes.describe()
# replacing 0 values to Nan values.
Diabetes_copy = Diabetes.copy(deep = True)

Diabetes[['Glucose', 'BloodPressure','SkinThickness', 'Insulin', 'BMI']] = Diabetes[['Glucose', 'BloodPressure','SkinThickness', 'Insulin', 'BMI']].replace(0, np.nan)
print(Diabetes.isnull().sum())
Diabetes.describe()
# np.nan values replaced by mean or median.
Diabetes['Glucose'].fillna(Diabetes['Glucose'].mean(), inplace = True)
Diabetes['BloodPressure'].fillna(Diabetes['BloodPressure'].mean(), inplace = True)

Diabetes['SkinThickness'].fillna(Diabetes['SkinThickness'].median(), inplace = True)
Diabetes['Insulin'].fillna(Diabetes['Insulin'].median(), inplace = True)
Diabetes['BMI'].fillna(Diabetes['BMI'].median(), inplace = True)
Diabetes.isnull().sum()
Diabetes.describe()
# DONE<jupyter_output><empty_output><jupyter_text>Visualizations<jupyter_code># For curve fittng 
from scipy import stats 
% matplotlib inline 
Diabetes.columns
# 1. Pregnancies 
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['Pregnancies'], kde = False, bins = 50)
plt.xlabel("Pregnancies")
plt.ylabel("Number of people")
plt.title("Number 0f pregnancies per person")

plt.savefig("Plots/Number of pregnacncies per person.png")
Diabetes['Pregnancies'].describe()
# most frequent case is 1 children per pima women.
# 2. Glucose 
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['Glucose'], kde = False, bins = 50)
plt.xlabel('Glucose')
plt.ylabel('Number of people')
plt.title("Glucose per person")

plt.savefig("Plots/Glucose per person.png")
# 3. BloodPressure 
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['BloodPressure'], kde = False, bins = 50)
plt.xlabel("Blood Pressure")
plt.ylabel("Number of people")
plt.title("Blood Pressure per person")

plt.savefig('Plots/Blood Pressure per person.png')
# for glucose and blood pressure, data seems to be of normal distribution.
# 4. SkinThickness
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['SkinThickness'], kde = False, bins = 50)
plt.xlabel('Skin Thickness')
plt.ylabel('No of people')
plt.title('Skin Thickness per person')

plt.savefig('Plots/Skin Thickness per person.png')
Diabetes.SkinThickness.describe()
# is normal , but contains 227 null values replaced by median
# 5. Insulin
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['Insulin'], kde = False, bins = 50)
plt.xlabel('Insulin')
plt.ylabel('No of People')
plt.title('Insulin per people')
plt.savefig("Plots/Insulin per people.png")
# 6. BMI
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['BMI'], kde = False, bins = 50)
plt.xlabel('BMI')
plt.ylabel('No of People')
plt.title("BMI per people")

plt.savefig('Plots/BMI per people.png')
Diabetes['BMI'].describe()
# seems Normal distribution
# 7. DiabetesPedigreeFunction 
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['DiabetesPedigreeFunction'], kde = False, bins = 50)
plt.xlabel('DiabetesPedigreeFunction')
plt.ylabel('No of People')
plt.title("DiabetesPedigreeFunction per people")

plt.savefig('Plots/DiabetesPedigreeFunction per people.png')
# need to convert to normal distribution 
# 8. Age 
plt.figure(figsize = (12, 6))
sns.distplot(Diabetes['Age'], kde = False, bins = 50)
plt.xlabel('Age')
plt.ylabel('No of People')
plt.title("Age per people")

plt.savefig('Plots/Age per people.png')
Diabetes.Age.describe()
# might need to convert to normal distribution.
# 9. Outcome
plt.figure(figsize = (12, 6))
sns.countplot(x = 'Outcome', data = Diabetes)
plt.xlabel('Outcome')
plt.ylabel('Count')
plt.title('Frequency of outcome to diabetes')

plt.savefig('Plots/Frequency of outcome to diabetes.png')
plt.show()
# Ratio of population with diabetes 
print("Ratio of people with diabetes :", str(len(Diabetes[Diabetes['Outcome'] == 1])/ len(Diabetes)))
# inbalanced Dataset<jupyter_output><empty_output><jupyter_text>Bivariate visualizations<jupyter_code># Correlation between variables 
plt.figure(figsize = (12, 6))
Corr = Diabetes[Diabetes.columns].corr()
sns.heatmap(Corr, annot=True)
plt.savefig('Plots/correlation_between_variables.png', bbinches = 'tight')<jupyter_output><empty_output><jupyter_text>* Not strong correlations between different features. (No need to drop features)* Glucose has a strong correlation with Outcome.* Some obvious correlations --> {Skinthickness : BMI}, {Age : Pregnancies}  Whole Diabetes dataset<jupyter_code>Diabetes.hist(figsize = (12, 12))
plt.savefig("Plots/Whole Diabetes data distribution.png")
plt.show()
Diabetes.plot(kind= 'box' , subplots=True, layout=(3,3), sharex=False, sharey=False, figsize=(10,8))
plt.savefig('Plots/box-plot.png')<jupyter_output><empty_output><jupyter_text>Modelling<jupyter_code>from sklearn.model_selection import train_test_split 
from sklearn.model_selection import (GridSearchCV, StratifiedKFold)

from sklearn import metrics 
from sklearn import model_selection 
from sklearn.metrics import accuracy_score 

from sklearn.linear_model import LinearRegression 
from sklearn.linear_model import LogisticRegression
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC 
from sklearn.neighbors import KNeighborsClassifier 
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier 
from sklearn.ensemble import AdaBoostClassifier 

from sklearn.datasets import make_classification
# Modele with clean data 
y  = Diabetes['Outcome']
X = Diabetes.drop(['Outcome'], axis = 1 )
scoring = 'roc_auc'
seed = 7
models = []

models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('CART', DecisionTreeClassifier()))
models.append(('NB', GaussianNB()))
models.append(('RF', RandomForestClassifier()))
models.append(('SVM', SVC()))
models.append(("Adaboost", AdaBoostClassifier()))

print('Evaluation metric : '+ scoring)
results = []
names = []

for name, model in models:
    kfold = model_selection.KFold(n_splits =10, random_state = seed)
    cv_results = model_selection.cross_val_score(model, X, y, cv = kfold, scoring = scoring)
    results.append(cv_results)
    names.append(name)
    
    print("Algorithm :", name)
    print("CV Mean : ", cv_results.mean())
    print("--"*30)
ax = sns.boxplot(data = results)
ax.set_xticklabels(names)
# Best algorithms --> LR and LDA 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 3 )
# LogisticRegression model with clean Dataset 
print('LogisticRegression model with clean Dataset ')
print("--"*30)
model1 = LogisticRegression()
fit1 = model1.fit(X_train, y_train)
pred1 = model1.predict(X_test)
confusion = metrics.confusion_matrix(y_test, pred1)
TP = confusion[1, 1]
TN = confusion[0, 0]
FP = confusion[0, 1]
FN = confusion[1, 0]

print('ROC_AUC score : ', metrics.roc_auc_score(y_test, pred1))
print("--"*30)
print('Matrix confusion : ', "\n", metrics.confusion_matrix(y_test, pred1))
print("--"*30)
print('Model sensitivity : ', TP / float(FN + TP))
print("--"*30)
print('Model specificiy : ', TN / (TN + FP))
print("--"*30)
print('Model accuracy : ', metrics.accuracy_score(y_test, pred1))
print("--"*30)
print("Classification report :")
print(classification_report(y_test, pred1))
conf = confusion_matrix(y_test,pred1)
label = ["0","1"]
sns.heatmap(conf, annot=True, xticklabels=label, yticklabels=label)
# Linear Discriminant analysis with clean dataset 
print('Linear Discriminant analysis with clean dataset ')
print('--'*30)

model2 = LinearDiscriminantAnalysis()
fit2 = model2.fit(X_train, y_train)
pred2 = model2.predict(X_test)
confusion = metrics.confusion_matrix(y_test, pred2)
TP = confusion[1, 1]
TN = confusion[0, 0]
FP = confusion[0, 1]
FN = confusion[1, 0]

print('ROC_AUC score : ', metrics.roc_auc_score(y_test, pred2))
print('--'*30)
print('Matrix confusion : ', "\n", metrics.confusion_matrix(y_test, pred2))
print('--'*30)
print('Model specificity : ', TP / float(FN + TP))
print('--'*30)
print('Model sensitivity : ', TN / (TN + FP))
print('--'*30)
print("Model accuracy : ", metrics.accuracy_score(y_test, pred2))
print('--'*30)
print('CLassification report : ')
print(classification_report(y_test, pred2))
conf = confusion_matrix(y_test,pred2)
label = ["0","1"]
sns.heatmap(conf, annot=True, xticklabels=label, yticklabels=label)
# Scalling and mormalizing dataset 
from sklearn.preprocessing import StandardScaler 

columns = X.columns 
scaler = StandardScaler()
scaler.fit(X)
X_scaled = scaler.transform(X)
X = pd.DataFrame(X_scaled, columns = [columns])
X.head()
# Modelling with clean and scaled dataset 
from sklearn import model_selection
scoring = 'roc_auc'
seed = 7 
models = []
models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('CART', DecisionTreeClassifier()))
models.append(('NB', GaussianNB()))
models.append(('RF', RandomForestClassifier()))
models.append(('SVM', SVC()))
models.append(('AdaBoost', AdaBoostClassifier()))

print('Evaluation metrics : ', scoring)
results = []
names = []

for name, model in models:
    kfold = model_selection.KFold(n_splits = 10 , random_state = seed)
    cv_results =  model_selection.cross_val_score(model, X, y, cv = kfold, scoring = scoring)
    results.append(cv_results)
    names.append(name)
    
    print('Algorithm : ', name)
    print('Data clean and scaled CV mean', cv_results.mean())
    print('--'*30)
# Best models --> LR, LDA 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 3)
# Linear Discriminant analysis with clean dataset and scaled dataset 
print('Linear Discriminant analysis with clean dataset and scaled dataset ')
print('--'*30)

model3 = LinearDiscriminantAnalysis()
fit3 = model3.fit(X_train, y_train)
pred3 = model3.predict(X_test)
confusion = metrics.confusion_matrix(y_test, pred3)
TP = confusion[1, 1]
TN = confusion[0, 0]
FP = confusion[0, 1]
FN = confusion[1, 0]

print('ROC_AUC score : ', metrics.roc_auc_score(y_test, pred3))
print('--'*30)
print('Matrix confusion : ', "\n", metrics.confusion_matrix(y_test, pred3))
print('--'*30)
print('Model specificity : ', TP / float(FN + TP))
print('--'*30)
print('Model sensitivity : ', TN / (TN + FP))
print('--'*30)
print("Model accuracy : ", metrics.accuracy_score(y_test, pred3))
print('--'*30)
print('CLassification report : ')
print(classification_report(y_test, pred3))
conf = confusion_matrix(y_test,pred3)
label = ["0","1"]
sns.heatmap(conf, annot=True, xticklabels=label, yticklabels=label)
# LogisticRegresssion on cleaned and scaled datset 
print('LogisticRegression model with clean Dataset ')
print("--"*30)
model4 = LogisticRegression()
fit4 = model4.fit(X_train, y_train)
pred4 = model4.predict(X_test)
confusion = metrics.confusion_matrix(y_test, pred1)
TP = confusion[1, 1]
TN = confusion[0, 0]
FP = confusion[0, 1]
FN = confusion[1, 0]

print('ROC_AUC score : ', metrics.roc_auc_score(y_test, pred4))
print("--"*30)
print('Matrix confusion : ', "\n", metrics.confusion_matrix(y_test, pred4))
print("--"*30)
print('Model sensitivity : ', TP / float(FN + TP))
print("--"*30)
print('Model specificiy : ', TN / (TN + FP))
print("--"*30)
print('Model accuracy : ', metrics.accuracy_score(y_test, pred4))
print("--"*30)
print("Classification report :")
print(classification_report(y_test, pred4))<jupyter_output>LogisticRegression model with clean Dataset 
------------------------------------------------------------
ROC_AUC score :  0.708976157083
------------------------------------------------------------
Matrix confusion :  
 [[80 12]
 [28 34]]
------------------------------------------------------------
Model sensitivity :  0.548387096774
------------------------------------------------------------
Model specificiy :  0.869565217391
------------------------------------------------------------
Model accuracy :  0.74025974026
------------------------------------------------------------
Classification report :
             precision    recall  f1-score   support

          0       0.74      0.87      0.80        92
          1       0.74      0.55      0.63        62

avg / total       0.74      0.74      0.73       154
{'fraction_non_alphanumeric': 0.1701729106628242, 'fraction_numerical': 0.024063400576368875, 'mean_word_length': 4.960068699012452, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '535464', 'license': 'MIT', 'n_tokens_mistral': 5069, 'n_tokens_neox': 4550, 'n_words': 1091, 'path': 'Pima Indians Diabetes Database /PIma_Indians_Diabetes_Prediction.ipynb', 'repo_name': 'MCParab/Kaggle-Titanic', 'size': '13880'}