<jupyter_start><jupyter_text>***Defining the Question***Provided with the dataset from Nairobi Hospital, your are task to build a model that determines whether or not the patient's symptoms indicate that the patient has hypothyroid.  ***Metric For Success***The Metric of Sucess will be to find a decission tree model to determine whether or not a patient has hypothyroid based on the patient's symptoms.  ***Context***Hypothyroid is a condition in which your thyroid gland doesn't produce enough of certain crucial hormones. An example of the crucial hormone is thyroxin which plays vital roles in digestion, heart and muscle function, brain development and maintenance of bones.With that stated, Nairobi Hospital conducted a clinical camp to test for hypothyroidism.  ***Experimental Design***The experimental design for this project was in adherence to the CRISP-DM methodology.It will follow the CRISP-DM steps which are:1.Problem Understanding2.Data Understanding3.Data Preparation4.Modelling5.Evaluation  **Data Relevance**The health care dataset is relevant to build a model that determines whether or not the patient's symptoms indicate that the patient has hypothyroid. ***Import the libraries***<jupyter_code>#ILl first import the necessary libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
from sklearn.decomposition import PCA
from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV
from sklearn import metrics
from sklearn.preprocessing import LabelEncoder, StandardScaler, MinMaxScaler
from sklearn.ensemble import RandomForestClassifier, \
GradientBoostingClassifier, AdaBoostClassifier
from sklearn.svm import SVC
from sklearn.tree import export_graphviz
from sklearn.externals.six import StringIO
from IPython.display import Image
import pydotplus
import warnings
warnings.filterwarnings('ignore') 
#ILl then load the dataset
df = pd.read_csv('http://bit.ly/hypothyroid_data')
df.head()
#Ill then check the tail of the dataset
df.tail()
#Ill check the the shape of the dataset
df.shape
# The output is 3163 rows and 26 columns
# Ill then use the .info() function which prints information about a DataFrame 
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 3163 entries, 0 to 3162
Data columns (total 26 columns):
 #   Column                     Non-Null Count  Dtype 
---  ------                     --------------  ----- 
 0   status                     3163 non-null   object
 1   age                        3163 non-null   object
 2   sex                        3163 non-null   object
 3   on_thyroxine               3163 non-null   object
 4   query_on_thyroxine         3163 non-null   object
 5   on_antithyroid_medication  3163 non-null   object
 6   thyroid_surgery            3163 non-null   object
 7   query_hypothyroid          3163 non-null   object
 8   query_hyperthyroid         3163 non-null   object
 9   pregnant                   3163 non-null   object
 10  sick                       3163 non-null   object
 11  tumor                      3163 non-null   object
 12  lithium                    3163 non-null   object
 13  goitre                     3163 non-null   object
 14  TSH_meas[...]<jupyter_text>The .info() function shows index dtype and column dtypes, non-null values and memory usage.  ***Data Cleaning***<jupyter_code>#ILl then ensure uniformity in my data by changing the column names to lowercase
df.columns = map(str.lower,df.columns)

#ILl then change the column names to small leter
df.columns
#Ill then check for nulls in the dataset
df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Output shows no null values<jupyter_code>#Ill then check for unique values in the dataset
#ILl use the for loop function to iterate through the column names
for column in df.columns:
  print(column)                                        
  print(df[column].unique())                           
  print("Their are", df[column].nunique(),"unique values in this column")
  #spacing purposes the "********"    
  print("***************************************************")<jupyter_output>status
['hypothyroid' 'negative']
Their are 2 unique values in this column
***************************************************
age
['72' '15' '24' '77' '85' '64' '20' '42' '69' '75' '53' '59' '68' '50'
 '78' '65' '28' '43' '40' '84' '?' '83' '63' '73' '36' '27' '48' '71' '60'
 '34' '79' '62' '74' '32' '41' '70' '29' '5' '58' '35' '51' '22' '16' '44'
 '39' '47' '18' '17' '88' '56' '30' '45' '66' '52' '54' '80' '33' '38'
 '92' '67' '46' '13' '89' '61' '49' '26' '57' '31' '90' '81' '25' '37'
 '76' '87' '23' '55' '86' '98' '21' '97' '1' '6' '14' '82' '11' '19' '9'
 '12' '10' '8' '7' '4' '93']
Their are 93 unique values in this column
***************************************************
sex
['M' 'F' '?']
Their are 3 unique values in this column
***************************************************
on_thyroxine
['f' 't']
Their are 2 unique values in this column
***************************************************
query_on_thyroxine
['f' 't']
Their are 2 unique values in this column
*************[...]<jupyter_text>After checking the unique values, I have noticed character "?" in some the column names. I'll convert them to nulls so that I impute them<jupyter_code>#Ill then replace the "?" with nan
df = df.replace(['?'], np.nan)
#Ill then check for nulls again
df.isnull().sum()
#Ill then drop the "tbg" column since it has many null values
df.drop(['tbg', 'tbg_measured'], axis = 1, inplace = True)
# ILl thenconfirm if it has been dropped
df.head()
#They are dropped
# Ill then check the datatypes again 
df.dtypes<jupyter_output><empty_output><jupyter_text>The output shows that all the column names are of object (string). Ill fix that into appropriate data types<jupyter_code># Ill then  use labelencoder because the categorical columns mostly have two
# initiating the label encoder object
labelenc = LabelEncoder()
df['status'] = labelenc.fit_transform(df['status'])

# replacing string values with numbers 0 and 1 
# f/false is transformed to 0 and t/true is transformed to 1,
# y/yes is transformed to 1 and n/no is transformed to 0. 
df['sex'].replace({'M':0, 'F':1}, inplace=True)
df.replace({'f':0,'t':1, 'y':1, 'n':0}, inplace=True)

# previewing the first five observations of the dataset
df.head()<jupyter_output><empty_output><jupyter_text>***Meaning***In the above output:1. 0 represents hypothyroidism and 1 represents negative.2.0 represents male and 1 represents females.3.false represents 0 and true represents 1,4.yes represents 1 and no represents 0.<jupyter_code>#ILl then confirm if the data types have been changed
df.dtypes<jupyter_output><empty_output><jupyter_text>***Imputation***Inorder to choose the appropriate imputation procedure,their are things to consider like knowing your continuos variable and categorical variable<jupyter_code>#Ill first create a copy for imputation
df_copy = df.copy(deep=True)
df_copy.head()
#Ill then convert columns of object datatype to integer datatype using pd.to_numeric function
convert = df_copy.columns[df_copy.dtypes == object]

df_copy[convert]= df_copy[convert].apply(pd.to_numeric, errors='coerce')

# Ill then confirm if the change has been effected
df_copy.dtypes
#ILl then do a value count so as to find the number of occurence in a dataframe
df_copy['sex'].value_counts()<jupyter_output><empty_output><jupyter_text>**Output reads 2255 females and 908 males****For the Categorical variable, imputing with the mode is generally used.**<jupyter_code>#Ill then fix the dataset. Inorder to start  imputing ill have to specify the intergers and strings so
# as to avoid 'can only concatenate str (not "int") to str'. 
# Dealing with the null values
numerical_columns = ['age', 'tsh', 't4u', 't3', 'tt4', 'fti']

# filling missing values in columns in mean_col list with mean values of each column
for col in numerical_columns:
  df_copy[col].fillna(df_copy[col].mean(), inplace=True)
df_copy['sex'].fillna(1.0, inplace = True)

# checking for null values after imputing
df_copy.isnull().sum()<jupyter_output><empty_output><jupyter_text>**I used a 1.0 (fillna)in sex because it was the most repeated number in the number.** Output reads 2255 females and 908 malesFor the Categorical variable, imputing with the mode is generally used.<jupyter_code>#ILl then confirm if their are still null values.
df_copy.isnull().sum()<jupyter_output><empty_output><jupyter_text>***Conclusion***On Imputing, I had to differetiate between categorical variables and continuos variables. Mean and Median are generally used in continuos variables(numeric) while categotical variables we use mode(most repeated)  ***Exploratory Data Analysis***We can define EDA as the process of performing initial investigation to data so as to unravel patterns, spot anomalies, test hypothesis and to check assumptions  ***Univariate Analysis***<jupyter_code>#Ill first check for outliers in the dataset with box plot
column_1 = ['age', 't3', 'tsh', 'fti', 't4u', 'tt4']

#Ill then  craft a list of colors for the boxplots
colors = ['cyan','magenta','yellow','black','blue','green']

# plotting the boxplots
figgg, axes = plt.subplots(2,3, figsize=(18,9))
plt.suptitle('Boxplots Showing Outliers in Numerical Columns', fontsize=18, color='darkblue', y=0.93)
#Ill then use a for loop for iterartion
for ax, data, col, color in zip(axes.flatten(), df_copy, column_1, colors):
  sns.boxplot(df_copy[col], ax=ax, color=color)
  ax.set_facecolor('antiquewhite')
plt.show();<jupyter_output><empty_output><jupyter_text>**I will not the outliers as they will take a huge percentage of the project.**<jupyter_code>#Ill then plot a bar graph to show the distribution in the sec column
df_copy.sex.value_counts().plot(kind='bar')
plt.xticks((np.arange(0, 2,)), ('Female', 'Male'), rotation=360)
plt.title('Number of Men and Women in the dataset', fontsize=15, color='black')
plt.show()
#Females are more
#Ill then plot a histogram to show distribution of the patients age
df_copy.age.plot.hist(grid=False, bins=20, rwidth=0.9,
                   color='blue')
plt.title('Age distribution of patients')
plt.xlabel('Age')
plt.ylabel('Number')
plt.grid(axis='y', alpha=0.75)<jupyter_output><empty_output><jupyter_text>**The output shows that their is a high number of patients between 50 and 60**  ***Bivariate Analysis***Bivariate analysis involves two different variables. It aims to find out the causes and relationships between those two variables.<jupyter_code>#Ill first preview my dataset
df_copy.head()
#Ill first check the overall correlation of the  columns
corr = df_copy.corr()
corr
#Ill then a plot a heatmap to show the coorelation of the variables
correlation= df_copy.corr() 
plt.figure(figsize = (20,10))
sns.heatmap(correlation, xticklabels=correlation.columns, yticklabels=correlation.columns, annot=True)
plt.title('A Heatmap of Patient Correlation in our Dataset', color='black')
plt.show()<jupyter_output><empty_output><jupyter_text>***Conclusion***1. Problem that were encountered was the pairplot was not running 2. From the above EDA, I chose not to drop the outliers for this project cause it will take a high percentage of data.3. I also found out that most patients were Females.4. Most patients were from the age 50 and 60.  ***Modelling***  ***Random Forest Classifier***<jupyter_code>#Ill first seperate the depenent variable and the independent variable
independent = df_copy.drop('status', 1)
dependent = df_copy['status']
#ILl then  split the data into training and test sets.
X_train, X_test, y_train, y_test = train_test_split(independent, dependent, test_size=.25, random_state=34)
#Ill then print out the shape of the train set and test set
print('X_train shape: ', X_train.shape, '\nY_Train Shape:', y_train.shape)
print('X_test  shape: ', X_test.shape, '\nY_Test Shape:', y_test.shape)
# Ill then scale the train and test sets using stadard scaler
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Ill then instantiate the random forest classifier object
rfc = RandomForestClassifier(n_estimators=100, random_state=42, min_samples_split=20, max_depth=5)
rfc = rfc.fit(X_train, y_train)
rfc_pred = rfc.predict(X_test)
#Ill check my models predictions
greg_Predicted = pd.DataFrame({'Actual': y_test, 'Predicted': rfc_pred})
greg_Predicted.head()
# ILl then print out the 
print('Mean Absolute Error:', metrics.mean_absolute_error(y_test, rfc_pred))  
print('Mean Squared Error:', metrics.mean_squared_error(y_test, rfc_pred))  
print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y_test, rfc_pred)))
print('R2 squared:', metrics.r2_score(y_test, rfc_pred)) 
print('Accuracy Score:', metrics.accuracy_score(y_test, rfc_pred))<jupyter_output>Mean Absolute Error: 0.008849557522123894
Mean Squared Error: 0.008849557522123894
Root Mean Squared Error: 0.09407208683835973
R2 squared: 0.7494343379491357
Accuracy Score: 0.9911504424778761<jupyter_text>***The accuracy of this model shows a 99% accuracy,It could be overfitting***<jupyter_code># lll then plott the decision trees from the random forest
dot_data = StringIO()
test_features = df_copy.columns.to_list()
test_features.remove('status')
tree = rfc.estimators_[50]

export_graphviz(tree, out_file=dot_data, filled=True, rounded=True, \
                special_characters=True, feature_names=test_features)
graph = pydotplus.graph_from_dot_data(dot_data.getvalue())
graph.write_png('hypothyroidism_forest.png')
Image(graph.create_png())<jupyter_output><empty_output><jupyter_text>***Conclusion***The plot shows the most important features in predicting whether a person is hypothyroid or not.**. the most import feature is fti test which accounts for 76% of the total feature importance**  ***Gradient Boosting Classifier***<jupyter_code>#ILl then  split the data into training and test sets.
X_train, X_test, y_train, y_test = train_test_split(independent, dependent, test_size=.25, random_state=34)

# Ill then scale the train and test sets using MinMaxScaler
mscaler = MinMaxScaler()
X_train = mscaler.fit_transform(X_train)
X_test = mscaler.transform(X_test)
#Ill first create  a list so as to get the best learning rate
list = [.05, .075, .1, .25, .5, .75, 1]

for x in list:
  gboost = GradientBoostingClassifier(n_estimators=100, learning_rate=x, \
                                      max_features=2, max_depth=5, random_state=42)
  gboost.fit(X_train, y_train)

  print('Learning rate: ', x)
  print('Training set accuracy score: {0:.3f}' .format(gboost.score(X_train, y_train)))
  print('Test set accuracy score: {0:.3f}' .format(gboost.score(X_test, y_test)))

  

# The output shows the best learing rate is 0.25
#ILl then get the accuracy score of the model
gb_clf = GradientBoostingClassifier(n_estimators=20, learning_rate=0.25, \
                                  max_features=2, max_depth=5, random_state=42)
gb_clf.fit(X_train, y_train)
gb_pred = gb_clf.predict(X_test)
print('\nAccuracy Score: ', metrics.accuracy_score(y_test, gb_pred))<jupyter_output><empty_output><jupyter_text>***ADA Boost Classifier***<jupyter_code>#ILl then  split the data into training and test sets.
X_train, X_test, y_train, y_test = train_test_split(independent, dependent, test_size=.25, random_state=34)
# ILl then instantiate the ada boost classifier object

ADA = AdaBoostClassifier(n_estimators=50, learning_rate=1)
ADA_boost= ADA.fit(X_train, y_train)
ADApred = ADA_boost.predict(X_test)
# ill then get the accuracy of the model
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

print('Ada Boost Classifier Accuracy: ', round(metrics.accuracy_score(y_test, ADApred) * 100, 2),'%')<jupyter_output>Ada Boost Classifier Accuracy:  99.12 %<jupyter_text>***Support Vector Machines***<jupyter_code># Ill then plot data and regression model
plt.figure(figsize=(10,8))
sample_hypo = df_copy.sample(300, random_state=308)
sns.lmplot('tsh', 'tt4', data=sample_hypo, hue='status', legend=False, palette='Set1', fit_reg=False, scatter_kws={'s': 70})
plt.legend({'Hypothroidism': 0, 'Negative': 1})
plt.title('TSH and TT4 test as the variable is equal to Status')
plt.show();<jupyter_output><empty_output><jupyter_text>***Fitting the Model***<jupyter_code># Ill then input the model
fitting= sample_hypo[['tsh', 'tt4']].values
# specifying label for the model
label_type = sample_hypo['status'].values
X_train, X_test, y_train, y_test = train_test_split(fitting, label_type, test_size=.25, random_state=34)
# fitting the model
svmodel = SVC(kernel = 'linear')
svclass = svmodel.fit(X_train, y_train)
# Create a mesh grid for our graph
def make_meshgrid(x, y, h=.02):
    x_min, x_max = x.min() - 1, x.max() + 1
    y_min, y_max = y.min() - 1, y.max() + 1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
    return xx, yy

# A function that creates the hyperplanes from our model
def plot_contours(ax, clf, xx, yy, **params):
    Z = svclass.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    out = ax.contourf(xx, yy, Z, **params)
    return out

fig, ax = plt.subplots()
# title for the plots
title = ('Hypothyroid Classification with linear SVC')
# Set-up grid for plotting.
t_sam = np.random.choice(fitting[:, 0], 300)
t_sam2 = np.random.choice(fitting[:, 1], 300)
X0, X1 = t_sam,t_sam2
xx, yy = make_meshgrid(X0, X1)

plot_contours(ax, svclass, xx, yy, cmap=plt.cm.coolwarm, alpha=0.8)
ax.scatter(X0, X1, c=label_type, cmap=plt.cm.coolwarm, s=20, edgecolors='k')
# ax.set_ylabel('TT4')
# ax.set_xlabel('TSH')
ax.set_xticks(())
# ax.legend(['Hypothyroid', 'Negative'])
ax.set_yticks(())
ax.set_title(title)
plt.show()
# Ill first take a sample of the dataframe
sample= df_copy.sample(300, random_state=308)

# input for the model
sample_feat = sample[['tsh', 'tt4']].values
# specifying label for the model
katono = sample['status'].values

sample_feat.shape

X_train, X_test, y_train, y_test = train_test_split(sample_feat, katono, test_size=.25, random_state=34)
model3 = SVC(kernel='poly', degree=3, gamma='auto', random_state=308)

# Ill then train the model usin the training sets
model3.fit(X_train, y_train)
# visualizing the data points in the kernel
# Create a mesh grid for our graph
def make_meshgrid(x, y, h=.02):
    x_min, x_max = x.min() - 1, x.max() + 1
    y_min, y_max = y.min() - 1, y.max() + 1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
    return xx, yy

# A function that creates the hyperplanes from our model
def plot_contours(ax, clf, xx, yy, **params):
    Z = model3.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    out = ax.contourf(xx, yy, Z, **params)
    return out

fig, ax = plt.subplots()
# title for the plots
title = ('Flower Classification with polynomial SVC')

# Set-up grid for plotting.
# pol_sam = np.random.choice(polfeat[:, 0], 300)
# pol_sam2 = np.random.choice(polfeat[:, 1], 300)
X0, X1 = sample_feat[:, 0], sample_feat[:, 1]
xx, yy = make_meshgrid(X0, X1)

plot_contours(ax, model3, xx, yy, cmap=plt.cm.Paired, alpha=0.8)
ax.scatter(X0, X1, c=poltar, cmap=plt.cm.Paired, s=20, edgecolors='k')
# ax.set_ylabel()
# ax.set_xlabel()
ax.set_xticks(())
ax.set_yticks(())
ax.set_title(title)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1263212168084558, 'fraction_numerical': 0.032585717968548596, 'mean_word_length': 4.936945209672483, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '6354', 'license': 'MIT', 'n_tokens_mistral': 6849, 'n_tokens_neox': 6268, 'n_words': 2088, 'path': 'Moringa_School_IPWeek_8_2021_Greg_Katono (1).ipynb', 'repo_name': 'Katonokatono/Decission-Trees-SVM', 'size': '19395'}