<jupyter_start><jupyter_text>FINAL PROJECT for CS 634  Name: <NAME> Topic: Predicting whether an individual is obese or not  based on their eating habits and physical condition  Github link: https://github.com/vac38/Classification_of_obesity.git Link to dataset: https://archive.ics.uci.edu/ml/datasets/Estimation+of+obesity+levels+based+on+eating+habits+and+physical+condition+<jupyter_code># Importing libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import scipy.stats as stats

from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import MinMaxScaler
# Storing data in a pandas dataframe 
ObesityData = pd.read_csv("/Users/veena/Desktop/DM_final_project/ObesityDataSet.csv")<jupyter_output><empty_output><jupyter_text>Exploratory Data analysis<jupyter_code># To display the data type for each feature/Atrribute
ObesityData.info()
ObesityData<jupyter_output><empty_output><jupyter_text>1) Renaming columns in data<jupyter_code>#Renaming columns in data
ObesityData.columns = ['Gender', 'Age', 'Height', 'Weight', 'family_history_with_overweight',
       'high_caloric_food', 'vegetables_consumption', 'main_meals', 'food_between_meals', 'SMOKE', 'Daily_water', 'Calories_consumption', 'physical_activity', 'technology_devices',
       'Alcohol_consumption', 'Transportation_used', 'Obesity']<jupyter_output><empty_output><jupyter_text>2) converting label values to binarySince the task for this project is to perform binary classification , the labels were categorized in to Normal or Obese using the following distinction: Insufficient Weight, Normal Weight, Overweight Level I, Overweight Level II → Categorized as ‘NORMAL’Obesity Type II and Obesity Type III →  categorized as ‘OBESE’<jupyter_code># Get all values present in the label column of dataset
ObesityData['Obesity'].unique()
# convert to labels to Normal and Obese
ObesityData['Obesity'].replace({'Normal_Weight': 'Normal','Overweight_Level_I':'Normal' , 'Overweight_Level_II':'Normal', 'Insufficient_Weight':'Normal', 'Obesity_Type_I':'Obese','Obesity_Type_II':'Obese','Obesity_Type_III':'Obese'}, inplace= True)
# Only two labels: Normal and Obese
ObesityData['Obesity'].unique()
# Checking for imbalance in data 
ObesityData['Obesity'].value_counts()<jupyter_output><empty_output><jupyter_text>The distribution of each class with the labels shows that the data is not balanced since 1139 records belong to ‘Normal’ class and 972 to ‘Obese’ class and their ratio is ~1.17  3) Shape of Data<jupyter_code>ObesityData.shape<jupyter_output><empty_output><jupyter_text>4) Check for null values<jupyter_code>#Check if there are any missing values
print("Column wise missing values in Data\n",ObesityData.isnull().sum())
sns.heatmap(ObesityData.isnull(), yticklabels=False)<jupyter_output>Column wise missing values in Data
 Gender                            0
Age                               0
Height                            0
Weight                            0
family_history_with_overweight    0
high_caloric_food                 0
vegetables_consumption            0
main_meals                        0
food_between_meals                0
SMOKE                             0
Daily_water                       0
Calories_consumption              0
physical_activity                 0
technology_devices                0
Alcohol_consumption               0
Transportation_used               0
Obesity                           0
dtype: int64<jupyter_text>5) Age group of people in Dataset<jupyter_code>sns.displot(ObesityData['Age'] , bins = 20, kde=True)
print('Average age: ',ObesityData['Age'].mean())<jupyter_output>Average age:  24.312599908574136<jupyter_text>The Age group of most of the participants in this study is 15 to 28 years with average age of 24 years  6) Average height and weight for the males and females<jupyter_code>sns.set()
fig = plt.figure(figsize=(20,10))
plt.subplot(1, 2, 1)
sns.boxplot(x='Gender', y='Height', data=ObesityData)
plt.subplot(1,2, 2)
sns.boxplot(x='Gender', y='Weight', data=ObesityData)<jupyter_output><empty_output><jupyter_text>The above box plots show that average height for males is more than females.Average weight of males is more than that of females  7) Relation plot for weight ,height , genders and obesity<jupyter_code>subdf1 = ObesityData.iloc[:,[0,2,3,16]]
sns.relplot(x="Height", y="Weight", hue="Obesity",style="Gender", data=subdf1)<jupyter_output><empty_output><jupyter_text>Th above plot shows how height and weight influence obesity. 1) People with higher weights tend to be more obese 2) Obesity does determined by ratio of height and weight.  Data Preprocessing  1) Label EncodingSince Classifiers cannot handle label data directly, label encoding is used.<jupyter_code>ObesityData.head(10)
lenc = LabelEncoder()
ObesityData['food_between_meals'] = lenc.fit_transform(ObesityData['food_between_meals'])
ObesityData['SMOKE'] = lenc.fit_transform(ObesityData['SMOKE'])
ObesityData['Calories_consumption'] = lenc.fit_transform(ObesityData['Calories_consumption'])
ObesityData['Alcohol_consumption'] = lenc.fit_transform(ObesityData['Alcohol_consumption'])
ObesityData['Gender'] = lenc.fit_transform(ObesityData['Gender'])
ObesityData['family_history_with_overweight'] = lenc.fit_transform(ObesityData['family_history_with_overweight'])
ObesityData['high_caloric_food'] = lenc.fit_transform(ObesityData['high_caloric_food'])
ObesityData['Transportation_used'] = lenc.fit_transform(ObesityData['Transportation_used'])
ObesityData['Obesity'] = lenc.fit_transform(ObesityData['Obesity'])
ObesityData.head(10)<jupyter_output><empty_output><jupyter_text>2) Correlation between different features<jupyter_code>#Correlation matrix
ObesityData.corr()

#Correlation heatmap
plt.figure(figsize=(15,10))
sns.heatmap(ObesityData.corr(), annot = True)

# No two features are highly correlated<jupyter_output><empty_output><jupyter_text>3) Splitting the data in to features(X) and Labels(Y)<jupyter_code>X_n = ObesityData[['Gender', 'Age', 'Height', 'Weight', 'family_history_with_overweight',
       'high_caloric_food', 'vegetables_consumption', 'main_meals', 'food_between_meals', 'SMOKE', 'Daily_water', 'Calories_consumption', 'physical_activity', 'technology_devices',
       'Alcohol_consumption','Transportation_used']].values
Y = ObesityData['Obesity']<jupyter_output><empty_output><jupyter_text>4) Normalization of Data The range of values for each feature are different. For example weight ranges from 39 kgs to 173 kgs and gender has only two values: 0 and 1. Therefore to convert all feature values between 0 and 1 , normalization is performed.<jupyter_code>#returns a numpy array with normalized values for X
min_max_scaler = MinMaxScaler()
X = min_max_scaler.fit_transform(X_n)<jupyter_output><empty_output><jupyter_text>Machine Learning models<jupyter_code>from sklearn.model_selection import KFold
from sklearn.metrics import confusion_matrix
from sklearn import linear_model
from sklearn.ensemble import RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier
from numpy import mean
def calc_evaluation_metrics(TN,FP,FN,TP):

    # Sensitivity (recall o true positive rate)
    Sensitivity = TP/(TP+FN)
    # Specificity(true negative rate)
    Specificity = TN/(TN+FP) 
    # Precision(positive predictive value)
    Precision = TP/(TP+FP)
    # Error Rate
    Err = (FP + FN)/(TP + FP + FN + TN)
    # Negative predictive value
    NPV = TN/(TN+FN)
    # False positive rate
    FPR = FP/(FP+TN)
    # False Discovery Rate
    FDR = FP / (FP + TP)
    # False negative rate
    FNR = FN/(TP+FN)
    # Overall accuracy
    Accuracy = (TP+TN)/(TP+FP+FN+TN)
    #F1_score 
    F1_score = (2 * TP)/(2 *( TP + FP + FN))
    #Balanced Acuuracy(BACC)
    BACC = (Sensitivity + Specificity)/2
    #True Skills Statistics(TSS)
    TSS = (𝑇𝑃/(𝑇𝑃+𝐹𝑁)) - (𝐹𝑃/(𝐹𝑃+𝑇𝑁))
    #Heidke Skill Score (HSS)
    num = 2 * ((TP*TN)-(FP*FN))
    denom = ((𝑇𝑃 + 𝐹𝑁) * ((𝐹𝑁+𝑇𝑁)+(TP+FP))* (𝐹𝑃+𝑇𝑁))
    HSS = num / denom
    return Accuracy,Sensitivity, Specificity, Precision,F1_score, Err, NPV, FPR,FDR,FNR,BACC,TSS,HSS
def kfold_split(X,Y,train_index, test_index):
    X_train, X_test = X[train_index], X[test_index] 
    y_train, y_test = Y[train_index], Y[test_index]
    return X_train, X_test,y_train, y_test

def c_matrix (y_test, LR_pred, m, i):
    c_matrix=confusion_matrix(y_test, LR_pred).ravel()
    TN, FP, FN, TP = c_matrix[0],c_matrix[1], c_matrix[2],c_matrix[3]
    Accuracy,Sensitivity, Specificity, Precision,F1_score, Err, NPV, FPR,FDR,FNR,BACC,TSS,HSS = calc_evaluation_metrics(TN,FP,FN,TP)
    metrics = [m,i, Accuracy,Sensitivity, Specificity, Precision,F1_score, Err, NPV, FPR,FDR,FNR,BACC,TSS,HSS]
    return metrics

def logistic(X_train, X_test,y_train, y_test):
    model_LR = linear_model.LogisticRegression(multi_class='ovr', solver='liblinear')
    model_LR.fit(X_train, y_train)
    LR_pred = model_LR.predict(X_test)
    return LR_pred


def decision_tree(X_train, X_test,y_train, y_test):
    decisiontree_model = DecisionTreeClassifier(random_state=0)
    decisiontree_model.fit(X_train,y_train)
    dt_pred = decisiontree_model.predict(X_test)
    return dt_pred

def random_forest(X_train, X_test,y_train, y_test):
    randomforest_model = RandomForestClassifier(max_depth = 100,  max_features= 3, min_samples_leaf= 3)
    randomforest_model.fit(X_train,y_train)
    rt_pred = randomforest_model.predict(X_test)
    return rt_pred<jupyter_output><empty_output><jupyter_text>Training and testing three diffrent machine learning models: Logistic Reression, Decision Tree and Random Forest<jupyter_code>kf = KFold(n_splits=10,random_state=None, shuffle = True) 
model_acc_LR = []
model_acc_DT = []
model_acc_RF = []

# LR = pd.DataFrame(columns =['model','fold','Accuracy','Sensitivity', 'Specificity', 'Precision', 'F1_score','Error rate', 'Negative predictive value', 'False positive rate', 'False Discovery Rate', 'False negative rate', 'Balanced Accuracy', 'True Skill Statistics','Heidke Skill Score'])
i = 1
for train_index, test_index in kf.split(X):
    # Sets of train and test
    X_train, X_test,y_train, y_test = kfold_split(X,Y, train_index, test_index)
    # models and prediction
    LR_pred = logistic(X_train, X_test,y_train, y_test)
    DT_pred = decision_tree(X_train, X_test,y_train, y_test)
    RF_pred = random_forest(X_train, X_test,y_train, y_test)

    #Evaluation : Logistic regression
    metric_LR = c_matrix(y_test, LR_pred, 'Logistic Regression', i)
    model_acc_LR.append(metric_LR)
    
    #Evaluation : Decision Tree
    metric_DT = c_matrix(y_test, DT_pred, 'Decision Tree', i)
    model_acc_DT.append(metric_DT)
    
    #Evaluation : Random Forest
    metric_RF = c_matrix(y_test, RF_pred, 'Random Forest', i)
    model_acc_RF.append(metric_RF)

    i += 1  
# Storing Data in Datframe
LR_metrics = pd.DataFrame(model_acc_LR, columns =['model','fold','Accuracy','Sensitivity', 'Specificity', 'Precision', 'F1_score','Error rate', 'Negative predictive value', 'False positive rate', 'False Discovery Rate', 'False negative rate', 'Balanced Accuracy', 'True Skill Statistics','Heidke Skill Score'])
LR_metrics.loc['Mean'] = LR_metrics.mean()

DT_metrics = pd.DataFrame(model_acc_DT, columns =['model','fold','Accuracy','Sensitivity', 'Specificity', 'Precision', 'F1_score','Error rate', 'Negative predictive value', 'False positive rate', 'False Discovery Rate', 'False negative rate', 'Balanced Accuracy', 'True Skill Statistics','Heidke Skill Score'])
DT_metrics.loc['Mean'] = DT_metrics.mean()

RF_metrics = pd.DataFrame(model_acc_RF, columns =['model','fold','Accuracy','Sensitivity', 'Specificity', 'Precision', 'F1_score','Error rate', 'Negative predictive value', 'False positive rate', 'False Discovery Rate', 'False negative rate', 'Balanced Accuracy', 'True Skill Statistics','Heidke Skill Score'])
RF_metrics.loc['Mean'] = RF_metrics.mean()

# Results for logistic regression performed on Obesity data using 10-fold cross validation
LR_metrics
# Results for Decision tree performed on Obesity data using 10-fold cross validation

DT_metrics
# Results for Random forest performed on Obesity data using 10-fold cross validation

RF_metrics<jupyter_output><empty_output><jupyter_text>Deep Learning  LSTM<jupyter_code>from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Dropout
model_acc_lstm = []
i = 1
for train, test in kf.split(X):
    X_train, X_test = X[train], X[test] 
    y_train, y_test = Y[train], Y[test]
  # create model
    model = Sequential()
    model.add(LSTM(200, activation='relu',input_shape=(1,16)))
    model.add(Dense(100, activation='relu'))
    model.add(Dense(50, activation='relu'))
    model.add(Dense(25, activation='sigmoid'))
    model.add(Dense(1))
    # Compile model
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    # Fit the model
    X_train_new = X_train.reshape((X_train.shape[0],1, X_train.shape[1]))
    X_test_new = X_test.reshape((X_test.shape[0],1, X_test.shape[1]))
    model.fit(X_train_new,y_train, epochs = 100, batch_size = 32, verbose=0)    
    # predict on the model
    predval = model.predict(X_test_new).flatten()
    predval_new = np.where(predval > 0.5, 1, 0)
    #Evalute the model
    metric_lstm = c_matrix(y_test, predval_new, 'LSTM', i)
    model_acc_lstm.append(metric_lstm)
    i += 1
    
    
LSTM_metrics = pd.DataFrame(model_acc_lstm, columns =['model','fold','Accuracy','Sensitivity', 'Specificity', 'Precision', 'F1_score','Error rate', 'Negative predictive value', 'False positive rate', 'False Discovery Rate', 'False negative rate', 'Balanced Accuracy', 'True Skill Statistics','Heidke Skill Score'])
LSTM_metrics.loc['Mean'] = LSTM_metrics.mean()

# Results for LSTM performed on Obesity data using 10-fold cross validation

LSTM_metrics
lr = pd.DataFrame(LR_metrics.iloc[10:,2:])
dt = pd.DataFrame(DT_metrics.iloc[10:,2:])
rf = pd.DataFrame(RF_metrics.iloc[10:,2:])
lstm = pd.DataFrame(LSTM_metrics.iloc[10:,2:])
k = [lr,dt,rf,lstm]
ALL_models = pd.concat(k)
obesity_predictions = ALL_models.set_axis(['Linear Regression', 'Decision Tree', 'Random Forest','LSTM'], axis=0)<jupyter_output><empty_output><jupyter_text>Conclusion<jupyter_code>obesity_predictions<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11474952971504215, 'fraction_numerical': 0.013028635128544555, 'mean_word_length': 5.495022624434389, 'pattern_counts': {'":': 0, '<': 68, '<?xml version=': 0, '>': 69, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '62476', 'license': 'MIT', 'n_tokens_mistral': 5153, 'n_tokens_neox': 4775, 'n_words': 1144, 'path': 'Chaudhari_veena_finaltermproj.ipynb', 'repo_name': 'vac38/Classification_of_obesity', 'size': '14362'}