<jupyter_start><jupyter_text>Predicting Diabetes Using Tree-Based Methods The dataset I will be using in the following parts is the "Pima Indians Diabetes Dataset". This dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases.  Objective of Project The objective of the dataset is to diagnostically predict whether or not a patient has diabetes, based on certain diagnostic measurements included in the dataset. Several constraints were placed on the selection of these instances from a larger database. In particular, all patients here are females at least 21 years old of Pima Indian heritage. There are two data files: "Diabetes_training.csv" "Diabetes_Xtest.csv" Training dataset, "Diabetes_training.csv", contains 576 rows and 9 columns. This is the training set containing both of the features and the target. Test dataset, "Diabetes_Xtest.csv", contains 192 rows and 8 columns. This is the test set which only contains the features. Prediction task is to determine the test for diabetes.Both files have the following fields, except quality which is not available in "Diabetes_Xtest.csv". Features:1 - pregnant: Number of times pregnant2 - glucose: Plasma glucose concentration (glucose tolerance test)3 - pressure: Diastolic blood pressure (mm Hg)4 - triceps: Triceps skin fold thickness (mm)5 - insulin: 2-Hour serum insulin (mu U/ml)6 - mass: Body mass index (weight in kg/(height in m)^2)7 - pedigree: Diabetes pedigree function8 - age: Age (years) Target:diabetes: ('neg','pos')  Import Packages<jupyter_code>import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Load training data and test data<jupyter_code>df_train = pd.read_csv('Diabetes_training.csv')
df_test = pd.read_csv('Diabetes_Xtest.csv')<jupyter_output><empty_output><jupyter_text>Exploratory data analysis<jupyter_code># inspect first few rows of the data
df_train.head()
# check if there are any missing values in the data
df_train.isnull().sum()
# quick overview of statistics of data
df_train.describe()<jupyter_output><empty_output><jupyter_text>Some takeawaysfrom the data observed, we see that many of the features above have min of 0. This doesn't make sense for glucose, pressure, triceps, insulin, and mass. Let's take a deeper look into the data.<jupyter_code># Missing Or Unwanted 0 values 
featurelist = []
count_of_zero_list = []

for col in df_train:
    cnt = 0
    for i in df_train[col]:
        if i==0:
            cnt = cnt + 1
    if col!='Outcome':
        #print (col, "-", cnt)
        featurelist.append(col)
        count_of_zero_list.append(cnt)
        
objects = tuple(featurelist)
y_pos = np.arange(len(featurelist))
performance = count_of_zero_list
 
fig_size = plt.rcParams["figure.figsize"]
 
# Set figure width to 12 and height to 9
fig_size[0] = 11
fig_size[1] = 3

plt.bar(y_pos, performance, align='center', color='b', alpha=0.9)
plt.xticks(y_pos, objects)

plt.ylabel('Count of 0 values')
plt.title('Count of 0 values per feature')
plt.grid(True)

plt.show()<jupyter_output><empty_output><jupyter_text>We clearly see that some catagories have very high count of 0 values. We will be replacing 0 values with either mean or median values for specific columns. To get the mean or median, we will have to filter out the zero values from the data frame to get a more accurate number.<jupyter_code># filter out the values that are not zero
glucose_nonzero = df_train.loc[df_train['glucose'] != 0]
pressure_nonzero = df_train.loc[df_train['pressure'] != 0]
triceps_nonzero = df_train.loc[df_train['triceps'] != 0]
insulin_nonzero = df_train.loc[df_train['insulin'] != 0]
mass_nonzero = df_train.loc[df_train['mass'] != 0]

# Correcting missing values in glucose, pressure, triceps, insulin, mass
df_train['glucose'] = df_train['glucose'].replace(0,glucose_nonzero['glucose'].mean())
df_train['pressure'] = df_train['pressure'].replace(0,pressure_nonzero['pressure'].mean())
df_train['triceps'] = df_train['triceps'].replace(0,triceps_nonzero['triceps'].median())
df_train['insulin'] = df_train['insulin'].replace(0,insulin_nonzero['insulin'].median())
df_train['mass'] = df_train['mass'].replace(0,mass_nonzero['mass'].mean())

df_train.describe()<jupyter_output><empty_output><jupyter_text>CorrelationCheck for multicolineary<jupyter_code>plt.figure(figsize=(13,10))
sns.heatmap(df_train.corr(),annot=True, fmt = ".2f", cmap = "coolwarm")<jupyter_output><empty_output><jupyter_text>Split training data into X and y<jupyter_code>X, y = df_train.drop('diabetes', axis=1), df_train['diabetes']

# transfrom the target as binary values: "pos" is 1, "neg" is 0
y = y.map({'pos': 1, 'neg': 0})<jupyter_output><empty_output><jupyter_text>Standardize the features on training and test set<jupyter_code>from sklearn.preprocessing import StandardScaler

scaler = StandardScaler().fit(X.values)
X = pd.DataFrame(scaler.transform(X.values), columns=X.columns)
X_test = pd.DataFrame(scaler.transform(df_test), columns=df_test.columns)<jupyter_output><empty_output><jupyter_text>Splitting training data into training and validation set<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size = 0.3, random_state=20)<jupyter_output><empty_output><jupyter_text>Random ForestDecision Trees have low predictive power compared to other methods due to high variance. Random Forest increases prediction power at the expense of decreased interpretability.From sklearn.ensemble use RandomForestClassifier to build a classification model with default parameters<jupyter_code>from sklearn.ensemble import RandomForestClassifier

model_rf = RandomForestClassifier(random_state=30)
model_rf.fit(X_train, y_train)

y_pred = model_rf.predict(X_val)<jupyter_output><empty_output><jupyter_text>Design Accuracy FunctionsImplement my_precision( ), my_recall( ), my_accuracy( ) functions to gain insights on the predicted values from the random forest model with default parameters<jupyter_code>def my_precision(y_val, y_pred): #TP/(TP+FP)
    TP = len([val for i, val in enumerate(y_val) if y_pred[i] == 1 and val == 1])
    FP = len([val for i, val in enumerate(y_val) if y_pred[i] == 1 and val == 0])
    return TP/(TP+FP)

def my_recall(y_val, y_pred): #TP/(TP+FN)
    TP = len([val for i, val in enumerate(y_val) if y_pred[i] == 1 and val == 1])
    FN = len([val for i, val in enumerate(y_val) if y_pred[i] == 0 and val == 1])
    return TP/(TP+FN)

def my_accuracy(y_val, y_pred): #(TP+TN)/(TP+FP+FN+TN)
    TP = len([val for i, val in enumerate(y_val) if y_pred[i] == 1 and val == 1])
    TN = len([val for i, val in enumerate(y_val) if y_pred[i] == 0 and val == 0])
    FN = len([val for i, val in enumerate(y_val) if y_pred[i] == 0 and val == 1])
    FP = len([val for i, val in enumerate(y_val) if y_pred[i] == 1 and val == 0])
    return (TP+TN)/(TP+FP+FN+TN)<jupyter_output><empty_output><jupyter_text>Print accuracy, precision and recall for the predictions<jupyter_code>print('my accuracy: ', my_accuracy(y_val, y_pred))
print('my precision: ', my_precision(y_val, y_pred))
print('my recall: ', my_recall(y_val, y_pred))<jupyter_output>my accuracy:  0.791907514450867
my precision:  0.6333333333333333
my recall:  0.7307692307692307<jupyter_text>Plot ROC curve and obtain AUC for the validation predictions<jupyter_code># Plot the ROC curve by giving appropriate names for title and axes. 
from sklearn.metrics import roc_curve
import sklearn.metrics as metrics

probs = model_rf.predict_proba(X_val)
preds = probs[:,1]
fpr, tpr, _ = roc_curve(y_val, preds)
roc_auc = metrics.auc(fpr, tpr)
print('AUC: ', roc_auc)
plt.title('Receiver Operating Characteristic')
plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)
plt.legend(loc = 'lower right')
plt.plot([0, 1], [0, 1],'r--')
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()<jupyter_output><empty_output><jupyter_text>Hyperparameters"Model tuning" refers to model adjustments to better fit the data, which is separate from "fitting" or "training" the model.There are aspects of some models that are user specified. For example, when using a random forest (which is basically an ensemble of decision trees), it is probably a good idea to choose the right number of underlying trees. Too many and the model might overfit, and too few and the model might not be able to properly learn the data. Parameters such as these are referred to as "hyperparameters" or "free parameters", as the values for these are determined by the user and not the algorithm.A quick and efficient way to optimize hyperparameters is to perform Grid Search over different values of the parameters.<jupyter_code>from sklearn.model_selection import GridSearchCV

# In the below dictionary, fill in the list of values that you want to try out for each parameter
# Refer to the descriptions in sklearn.ensemble.RandomForestClassifier's doc to understand what the parameters depict

param_grid = {
    'max_depth': [7, 8, 9],
    'max_features': ['auto', 'sqrt', 'log2', 8],
    'min_samples_leaf': [5, 15, 25],
    'min_samples_split': [2, 5, 8],
    'n_estimators': [25, 30, 35]
}

model_rf = RandomForestClassifier(random_state=30)

# Fit on the training data with all sets of parameters.

grid_search = GridSearchCV(estimator = model_rf, scoring='recall', param_grid = param_grid, 
                          cv = 3, n_jobs = -1, verbose = 2)
# Fit on the training data with all sets of parameters.

grid_search.fit(X_train, y_train)
# best param
grid_search.best_params_
# Print the best score
grid_search.best_score_
### Using the best model, do the following:
### Make predictions on the training set and display accuracy, precision and recall.
### Make predictions on the validation set and display accuracy, precision and recall.
rf = RandomForestClassifier(**grid_search.best_params_, random_state=30)
rf.fit(X_train, y_train)
y_train_pred = rf.predict(X_train) 
y_preds = rf.predict(X_val) 

### Using the best model, do the following:
### Make predictions on the training set and display accuracy, precision and recall.
### WRITE CODE HERE ###
print('Make predictions on training set')
print('my accuracy: ', my_accuracy(y_val, y_preds))
print('my precision: ', my_precision(y_train, y_train_pred))
print('my recall: ', my_recall(y_train, y_train_pred))
print()

### Make predictions on the validation set and display accuracy, precision and recall. 
### WRITE CODE HERE ###
print('Make predictions on validation set')
print('my accuracy: ', my_accuracy(y_val, y_preds))
print('my precision: ', my_precision(y_val, y_preds))
print('my recall: ', my_recall(y_val, y_preds))
probs = rf.predict_proba(X_val)
preds = probs[:,1]
fpr, tpr, _ = roc_curve(y_val, preds)
roc_auc = metrics.auc(fpr, tpr)
print('AUC: ',roc_auc)
plt.title('Receiver Operating Characteristic')
plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)
plt.legend(loc = 'lower right')
plt.plot([0, 1], [0, 1],'r--')
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10879712746858168, 'fraction_numerical': 0.015350089766606822, 'mean_word_length': 5.65531660692951, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '588523', 'license': 'MIT', 'n_tokens_mistral': 3824, 'n_tokens_neox': 3535, 'n_words': 1188, 'path': 'decsiontree_diabetes.ipynb', 'repo_name': 'garethcheng/decisiontree_diabetes', 'size': '11140'}