<jupyter_start><jupyter_text>Import necessary libraries<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
import sklearn.metrics as metrics
from sklearn.metrics import accuracy_score
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score
from sklearn.metrics import confusion_matrix<jupyter_output><empty_output><jupyter_text>Load the Heart Failure Clinical Record Dataset<jupyter_code>heart_failure_df = pd.read_csv("heart_failure_clinical_records_dataset.csv")
heart_failure_df.info()
heart_failure_df.columns<jupyter_output><empty_output><jupyter_text>Let us observe the distribution of the continuous variables<jupyter_code>heart_failure_df[['age', 'creatinine_phosphokinase',
                 'ejection_fraction', 'platelets',
                 'serum_creatinine', 
                 'serum_sodium']].hist(bins=20,
                                       figsize=(15, 15))
plt.show()
continous_var = ['age', 'creatinine_phosphokinase',
                'ejection_fraction', 'platelets',
                'serum_creatinine', 'serum_sodium']

heart_failure_df[continous_var].describe()<jupyter_output><empty_output><jupyter_text>Now as we've seen the distribution separately and statistical parameters, it would be nice a idea to see how each of these variables relate to the 'DEATH_EVENT'. Lets replace the 0 and 1 values of this column to 'yes' and 'no' as well as change the 'sex' column values of 0 and 1 with 'male' and 'female'.<jupyter_code>heart_failure_df['sex1'] = heart_failure_df['sex'].replace({1: "Male", 0: "Female"})
heart_failure_df['death'] = heart_failure_df['DEATH_EVENT'].replace({1: "yes", 0: "no"})<jupyter_output><empty_output><jupyter_text>Let us observe the relationship between continuous variables<jupyter_code>sns.pairplot(heart_failure_df[["creatinine_phosphokinase", 
                               "ejection_fraction", "platelets", 
                               "serum_creatinine","serum_sodium", 
                               "time", "death"]], hue="death",
                                diag_kind="kde",
                                kind="scatter",
                                palette="husl")

plt.show()<jupyter_output><empty_output><jupyter_text>In the above plot, the red color represents the death event and the green color represents no death event. As such the plot shows us how each of these variables is segregated between death events. Besides, the scatter plot, the density plot between shows a clear distinction of data for death events and no death events. Boxplots will give us a little more clarity on that.<jupyter_code>continous_var = ['age', 'creatinine_phosphokinase', 
                 'ejection_fraction', 'platelets', 
                 'serum_creatinine','serum_sodium']

plt.figure(figsize=(16, 25))

for i, col in enumerate(continous_var):
    plt.subplot(6, 4, i*2+1)
    plt.subplots_adjust(hspace=0.25, wspace=0.3)
    
    plt.grid(True)
    plt.title(col)
    sns.kdeplot(heart_failure_df.loc[heart_failure_df["death"]=='no', col],
                label="alive", color="green", shade=True, kernel='gau',
                cut=0)
    sns.kdeplot(heart_failure_df.loc[heart_failure_df["death"]=='yes', col],
                label="dead", color="red", shade=True, kernel='gau',
                cut=0)
    plt.subplot(6, 4, i*2+2)
    sns.boxplot(y = col, data = heart_failure_df, x="death", palette = ["green","red"])<jupyter_output>/Users/jessica/anaconda3/lib/python3.7/site-packages/seaborn/distributions.py:1666: UserWarning: Support for alternate kernels has been removed. Using Gaussian kernel.
  warnings.warn(msg, UserWarning)
/Users/jessica/anaconda3/lib/python3.7/site-packages/seaborn/distributions.py:1666: UserWarning: Support for alternate kernels has been removed. Using Gaussian kernel.
  warnings.warn(msg, UserWarning)
/Users/jessica/anaconda3/lib/python3.7/site-packages/seaborn/distributions.py:1666: UserWarning: Support for alternate kernels has been removed. Using Gaussian kernel.
  warnings.warn(msg, UserWarning)
/Users/jessica/anaconda3/lib/python3.7/site-packages/seaborn/distributions.py:1666: UserWarning: Support for alternate kernels has been removed. Using Gaussian kernel.
  warnings.warn(msg, UserWarning)
/Users/jessica/anaconda3/lib/python3.7/site-packages/seaborn/distributions.py:1666: UserWarning: Support for alternate kernels has been removed. Using Gaussian kernel.
  warnings.warn(msg, Use[...]<jupyter_text>Next we will find the mean and median of each continuous variable for death events and no death events. As we saw in the distribution and density plots before, not all the variables are normally distributed. Some are skewed. So, mean alone will not be representative for each one.<jupyter_code>y = heart_failure_df.groupby("death")["creatinine_phosphokinase", 
                                      "ejection_fraction", 
                                      "platelets", 
                                      "serum_creatinine", 
                                      "serum_sodium", 
                                      "time"].agg([np.mean, np.median])
y<jupyter_output>/Users/jessica/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:1: FutureWarning: Indexing with multiple keys (implicitly converted to a tuple of keys) will be deprecated, use a list instead.
  """Entry point for launching an IPython kernel.<jupyter_text>Looks like the 'time' variable is very different between the death events. Let's see if high blood pressure plays any role in deaths by gender<jupyter_code>heart_failure_df.groupby(['sex1', 'high_blood_pressure', 'death']).size().unstack().fillna(0).apply(lambda x: x/x.sum(), axis=1)<jupyter_output><empty_output><jupyter_text>As we can see from the above observation that in the female population, 28% of deaths happened with no high blood pressure and 39% of deaths happened with high blood pressure. Apparently, a significant difference but not too drastic. Let's explore other five categorical variables in this dataset with help of countplot.<jupyter_code>binary_var = ['anaemia', 'diabetes', 'high_blood_pressure',
             'sex1', 'smoking']
plt.figure(figsize=(13, 9))
for i, var in enumerate(binary_var):
    plt.subplot(2, 3, i+1)
    plt.title(var, fontsize=14)
    plt.xlabel(var, fontsize=12)
    plt.ylabel("Count", fontsize=12)
    plt.subplots_adjust(hspace=0.4, wspace=0.3)
    sns.countplot(data=heart_failure_df, x=var, hue="death", 
                 palette=['gray','coral'])<jupyter_output><empty_output><jupyter_text>In the above plot, we can clearly see that there is a difference in the number of death events between different sex(male and female), high_blood_pressure, smoking, and diabetes status. But at the same time, it also shows that the dataset is not balanced in terms of the number of people who smoke and do not smoke or the number of people having diabetes or not, or the number of males and the number of females. At this moment, I can think of taking crosstab between each of these variables and death variable.<jupyter_code>x = pd.crosstab(heart_failure_df["sex1"], heart_failure_df["death"])
x<jupyter_output><empty_output><jupyter_text>We can see how many deaths happened in the male population versus the female population. But the proportions will be more informative in this case.<jupyter_code>x.apply(lambda z: z/z.sum(), axis=1)<jupyter_output><empty_output><jupyter_text>The deaths in male and female population both are approximately 32%. So they are almost identical.<jupyter_code>x = pd.crosstab(heart_failure_df["anaemia"], heart_failure_df["death"])
x
x.apply(lambda z: z/z.sum(), axis=1)<jupyter_output><empty_output><jupyter_text>The proportion of death for people with anemia is bit higher.<jupyter_code>x = pd.crosstab(heart_failure_df["diabetes"], heart_failure_df["death"])
x
x.apply(lambda z: z/z.sum(), axis=1)<jupyter_output><empty_output><jupyter_text>Here, the proportion of death for people with diabetes and with no diabetes is exactly the same.<jupyter_code>x = pd.crosstab(heart_failure_df["smoking"], heart_failure_df["death"])
x
x.apply(lambda z: z/z.sum(), axis=1)<jupyter_output><empty_output><jupyter_text>The proportion of death for people with smoking and with no smoking is exactly the same.<jupyter_code>x = pd.crosstab(heart_failure_df["high_blood_pressure"], heart_failure_df["death"])
x
x.apply(lambda z: z/z.sum(), axis=1)<jupyter_output><empty_output><jupyter_text>Here, the proportion of death is higher in people with high blood pressure. In all the above analyses, we only tried to determine all the other variable’s relationship with the ‘death’ variable. Next element is a violin plot that shows the distribution of ‘time’ across smoking and no smoking males and females.<jupyter_code>plt.figure(figsize=(8, 6))
a = sns.violinplot(heart_failure_df.smoking, heart_failure_df.time, 
                   hue=heart_failure_df.sex1, split=True)
plt.title("Smoking vs Time Segregated By Gender", fontsize=14)
plt.xlabel("Smoking", fontsize=12)
plt.ylabel("Time", fontsize=12)
plt.show()<jupyter_output>/Users/jessica/anaconda3/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>As we can observe from the above plot that in case of no smoking people, male and females have the same distribution, while on the other hand, in case of smoking people, the distribution for males and females is very different. Most females lie in a narrow range from about 0 to 140. Whereas the male population has a range from -50 to 350.  Observe the relation between Ejection_fraction and Time segregated by Death<jupyter_code>sns.lmplot(x="ejection_fraction", y="time",
          hue="death", data=heart_failure_df,
          scatter_kws=dict(s=40, linewidths=0.7,
                           edgecolors="black"))
plt.xlabel("Ejection Fraction", fontsize=12)
plt.ylabel("Time", fontsize=12)
plt.title("Ejection fraction vs time segregated by death", 
          fontsize=14)
plt.show()<jupyter_output><empty_output><jupyter_text>From the above plot we can observe the regression line and the confidence band. As expected the confidence band is narrower in the middle where the density of data is higer and wider in the sides where the density of data is lower.In the next plot, let's see another comparison between the male and female population. How 'time' changes with 'age':<jupyter_code>fig = plt.figure(figsize=(20, 8), dpi=80) 
g = sns.lmplot(x='age', y='time',
               data = heart_failure_df,
               palette="Set1", col="sex1", 
               scatter_kws=dict(s=60, linewidths=0.7,edgecolors="black")) 

for ax in g.axes.flat:
    ax.set_title(ax.get_title(), 
                 fontsize='x-large') 
    ax.set_ylabel(ax.get_ylabel(), 
                  fontsize='x-large') 
    ax.set_xlabel(ax.get_xlabel(), 
                  fontsize='x-large')<jupyter_output><empty_output><jupyter_text>If we notice the regression line, in case of male population it is much steeper. With 'age', 'time' goes down.  Let's understand the correlation between the variables - using a heatmap<jupyter_code>plt.figure(figsize=(10, 10))
sns.heatmap(heart_failure_df.corr(), annot=True, linewidths=0.5, 
            cmap="crest")
plt.show()<jupyter_output><empty_output><jupyter_text>Prediction of heart failure  Data PreparationIn this dataset all the columns were originally numeric but we had created two columns with string which must be deleted.<jupyter_code>heart_failure_df = heart_failure_df.drop(columns=['sex1','death'])
# In case if there are nay null values
heart_failure_df = heart_failure_df.dropna()<jupyter_output><empty_output><jupyter_text>As we can see from the above heatmap that the correlation between 'DEATH_EVENT' and 'anaemia', 'diabetes', 'sex', 'smoking', 'creatinine_phosphokinase' is very low, might be good idea to drop those columns.<jupyter_code>heart_failure_df = heart_failure_df.drop(columns=['anaemia',
                                                  'diabetes',
                                                  'sex',
                                                  'smoking',
                                                  'creatinine_phosphokinase'])<jupyter_output><empty_output><jupyter_text>It is good idea to bring all the variables in our dataset on the same scale before we train our model. Like 'platelets' have a very high range whereas 'serum_creatinine' has a very low range.<jupyter_code>heart_failure_df2 = heart_failure_df
continuous_var = ['age', 'ejection_fraction', 
                  'platelets', 'serum_creatinine', 
                  'serum_sodium']
for i in continuous_var:
    heart_failure_df2[i] = heart_failure_df2[i] / max(heart_failure_df2[i])
## Machine learning model will be used to predict DEATH_EVENT
## DEATH_EVENT will be our output variable.
y = heart_failure_df2['DEATH_EVENT']
## Rest of the variables will be used to train our model
X = heart_failure_df2.drop(columns=['DEATH_EVENT'])<jupyter_output><empty_output><jupyter_text>Separation of Training and Testing Data<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                   test_size=0.25,
                                                   random_state=22)<jupyter_output><empty_output><jupyter_text>Using Decision Tree Classifier<jupyter_code>clf_tree = DecisionTreeClassifier(random_state=21, max_depth = 7, max_leaf_nodes=6).fit(X_train,y_train)
y_pred = clf_tree.predict(X_test) 
print("Accuracy:",metrics.accuracy_score(y_test,y_pred)) 
print("Precision:",metrics.precision_score(y_test,y_pred,pos_label=0) )
print("Recall:",metrics.recall_score(y_test,y_pred,pos_label=0)) 
print("F Score:",metrics.f1_score(y_test,y_pred,pos_label=0)) 
print("Confusion Matrix:\n",metrics.confusion_matrix(y_test,y_pred))<jupyter_output>Accuracy: 0.8133333333333334
Precision: 0.9361702127659575
Recall: 0.8
F Score: 0.8627450980392157
Confusion Matrix:
 [[44 11]
 [ 3 17]]
{'fraction_non_alphanumeric': 0.09797109505280711, 'fraction_numerical': 0.01660644802668149, 'mean_word_length': 4.085865724381626, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 93, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '24', 'dir': 'jupyter-structured-clean-dedup', 'id': '315105', 'license': 'MIT', 'n_tokens_mistral': 4598, 'n_tokens_neox': 4256, 'n_words': 1242, 'path': 'Heart_Failure_Clinical_Record_Prediction.ipynb', 'repo_name': 'ochoa-jessica/Heart_Failure_Prediction', 'size': '14392'}