<jupyter_start><jupyter_text>Dimension of the dataset is 929 X 14, i.e. 929 rows and 14 features i.e. columns.<jupyter_code># Check continuous columns
con=df._get_numeric_data().columns
print("No of Continuous columns:",len(df._get_numeric_data().columns),"\n\n",con)
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 929 entries, 0 to 928
Data columns (total 14 columns):
 #   Column           Non-Null Count  Dtype 
---  ------           --------------  ----- 
 0   Age              929 non-null    int64 
 1   Sex              929 non-null    int64 
 2   ChestPain        929 non-null    int64 
 3   Rest_bp          929 non-null    object
 4   Cholestrol       929 non-null    object
 5   Fast_bld_sugar   929 non-null    object
 6   Rest_Ecg         929 non-null    object
 7   Max_Rt           929 non-null    object
 8   Ex_Angina        929 non-null    object
 9   St_Depr          929 non-null    object
 10  Slope            929 non-null    object
 11  Colored_Vessels  929 non-null    object
 12  Thalassemia      929 non-null    object
 13  Target           929 non-null    int64 
dtypes: int64(4), object(10)
memory usage: 101.7+ KB<jupyter_text>Only 4 columns are shown as contiuous columns, when in actuality all columns must be numeric, so there are certain values which make them categorical. To make sure what the values are in each column we would perform value_counts() function on each column and determine what exactly is making the columns categorical.  value_counts on each column<jupyter_code>col_name=list(df.columns)
col_name 
#These are all the columns in the dataset
for i in col_name:
    print(df[i].value_counts(),"\n")<jupyter_output>58    53
57    50
62    44
60    42
54    42
59    41
56    39
63    38
51    38
61    37
55    34
64    32
52    32
53    29
65    26
67    24
44    23
41    23
50    20
43    20
66    19
48    19
42    19
46    18
45    18
69    16
68    14
47    14
49    12
70    11
38    11
35    10
40     9
39     8
71     8
74     8
37     5
34     5
72     4
77     3
76     3
75     3
29     2
36     1
73     1
32     1
Name: Age, dtype: int64 

1    719
0    210
Name: Sex, dtype: int64 

4    517
3    236
2    118
1     58
Name: ChestPain, dtype: int64 

120    103
130     97
140     84
?       58
110     57
150     50
160     41
125     32
128     28
138     26
112     20
115     20
135     19
132     19
124     16
145     16
118     15
122     14
134     14
170     13
100     13
108     12
152     12
142     11
105     11
136     10
126     10
180      9
144      8
155      8
95       6
178      5
146      5
102      5
158      5
104      4
94       4
200      4
148      4
154      4
165     [...]<jupyter_text>After looking at the Value counts of each column we can see that there are ? in places where there are no values. So replace these ? special character with NaN's.<jupyter_code>df.isna().sum()
df=df.replace("?",np.NaN)
df.isna().sum()
df.describe(include="all") #All features included, categorical and continuous 
df.describe() # Only continuous values
# In this case it would be same as above since there are only continuous columns
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 929 entries, 0 to 928
Data columns (total 14 columns):
 #   Column           Non-Null Count  Dtype 
---  ------           --------------  ----- 
 0   Age              929 non-null    int64 
 1   Sex              929 non-null    int64 
 2   ChestPain        929 non-null    int64 
 3   Rest_bp          871 non-null    object
 4   Cholestrol       922 non-null    object
 5   Fast_bld_sugar   847 non-null    object
 6   Rest_Ecg         928 non-null    object
 7   Max_Rt           875 non-null    object
 8   Ex_Angina        875 non-null    object
 9   St_Depr          867 non-null    object
 10  Slope            810 non-null    object
 11  Colored_Vessels  605 non-null    object
 12  Thalassemia      707 non-null    object
 13  Target           929 non-null    int64 
dtypes: int64(4), object(10)
memory usage: 101.7+ KB<jupyter_text>Visualizing and Filling Missing Values<jupyter_code>import missingno as msn
msn.matrix(df)
msn.heatmap(df)
#missingno.heatmap visualizes the correlation matrix about the locations of missing values in columns.
msn.bar(df)<jupyter_output><empty_output><jupyter_text>The Nan's have to be filled, for filling the NaN valujes there are many methods to do so, but I am going to use mean (interpolate) and mode to fill the NaN values.After looking at the values of columns having NaN values, I have decided for the columns who have wide range of values I would use interpolate mean method to fill the NaN's and for columns who have very few range of values like here for the Fast blood sugar column we have 2 values 0 i.e. no high blood sugar and 1 i.e. has higher than normal range blood sugar, for such columns I would use the mode method to fill the NaN values.<jupyter_code>mode_col=['Fast_bld_sugar','Rest_Ecg','Ex_Angina','Slope','Colored_Vessels','Thalassemia']
for col in mode_col:
    df[col].fillna(df[col].mode()[0],inplace=True)
    
df.isna().sum()<jupyter_output><empty_output><jupyter_text>The above cell is for thode columns which are to be filled by mode. Now for the rest of the columns I will use the interpolate mean method to fill the NaN values.<jupyter_code>df.fillna(df.mean()[0],inplace=True)
df.isna().sum()
column_list = df.columns
for i in column_list:
    print("Values of",i,"column\n",df[i].unique())
    print("--------------\n")<jupyter_output>Values of Age column
 [63 67 37 41 56 62 57 53 44 52 48 54 49 64 58 60 50 66 43 40 69 59 42 55
 61 65 71 51 46 45 39 68 47 34 35 29 70 77 38 74 76 32 36 72 73 75]
--------------

Values of Sex column
 [1 0]
--------------

Values of ChestPain column
 [1 4 3 2]
--------------

Values of Rest_bp column
 ['145' '160' '120' '130' '140' '172' '150' '110' '132' '117' '135' '112'
 '105' '124' '125' '142' '128' '170' '155' '104' '180' '138' '108' '134'
 '122' '115' '118' '100' '200' '94' '165' '102' '152' '101' '126' '174'
 '148' '178' '158' '192' '129' '144' '123' '136' '146' '106' '156' '154'
 '114' '164' '95' 55.61248654467169 '80' '185' '116' '0' '96' '190' '127']
--------------

Values of Cholestrol column
 ['233' '286' '229' '250' '204' '236' '268' '354' '254' '203' '192' '294'
 '256' '263' '199' '168' '239' '275' '266' '211' '283' '284' '224' '206'
 '219' '340' '226' '247' '167' '230' '335' '234' '177' '276' '353' '243'
 '225' '302' '212' '330' '175' '417' '197' '198' '290' '253' '172' [...]<jupyter_text>Change the values of the object i.e. the categorical columns.The values will be assigned as given in the UCI site for each column.<jupyter_code>df['Sex'][df['Sex'] == 0] = 'female'
df['Sex'][df['Sex'] == 1] = 'male'

df['ChestPain'][df['ChestPain'] == 1] = 'typical angina'
df['ChestPain'][df['ChestPain'] == 2] = 'atypical angina'
df['ChestPain'][df['ChestPain'] == 3] = 'non-anginal pain'
df['ChestPain'][df['ChestPain'] == 4] = 'asymptomatic'

df['Fast_bld_sugar'][df['Fast_bld_sugar'] == '0'] = 'lower than 120mg/ml'
df['Fast_bld_sugar'][df['Fast_bld_sugar'] == '1'] = 'greater than 120mg/ml'

df['Rest_Ecg'][df['Rest_Ecg'] == '0'] = 'normal'
df['Rest_Ecg'][df['Rest_Ecg'] == '1'] = 'ST-T wave abnormality'
df['Rest_Ecg'][df['Rest_Ecg'] == '2'] = 'left ventricular hypertrophy'

df['Ex_Angina'][df['Ex_Angina'] == '0'] = 'no'
df['Ex_Angina'][df['Ex_Angina'] == '1'] = 'yes'

df['Slope'][df['Slope'] == '1'] = 'upsloping'
df['Slope'][df['Slope'] == '2'] = 'flat'
df['Slope'][df['Slope'] == '3'] = 'downsloping'

# Values of Colored_Vessels column
#  ['0' '3' '2' '1']
# ca: number of major vessels (0-3) colored by flourosopy

df['Thalassemia'][df['Thalassemia'] == '3'] = 'normal'
df['Thalassemia'][df['Thalassemia'] == '6'] = 'fixed defect'
df['Thalassemia'][df['Thalassemia'] == '7'] = 'reversable defect'

df['Target'][df['Target'] == 2] = 1
df['Target'][df['Target'] == 3] = 1
df['Target'][df['Target'] == 4] = 1
df.info()
df.head()
df=df.replace(-0.9,df.mean())
# In the UCI data repository it is mentioned that missing 
# values are distinguised with -0.9, so we replace this 
# value also with the mean value.
#Lets check for missing values once again
msn.bar(df,color='Purple')<jupyter_output><empty_output><jupyter_text>There are no missing values, now visualization of the data.  Visualization<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
import plotly.graph_objects as go
import plotly.express as px
import plotly.io as pio
total = len(df['Target'])
ax = sns.countplot(y="Target", data=df, palette="hls")
ax.set_title('Total percentage of people with and without Heart Disease')
plt.xlabel('No of people')

total = len(df['Target'])
for p in ax.patches:
        percentage = '{:.1f}%'.format(100 * p.get_width()/total)
        x = p.get_x() + p.get_width() + 0.02
        y = p.get_y() + p.get_height()/2
        ax.annotate(percentage, (x, y))

plt.show()<jupyter_output><empty_output><jupyter_text>Percentage of people with heart disease is 58.3% Percentage of people without heart disease is 41.7%<jupyter_code>df.sample(4)
total = len(df['Target'])
sns.set(style="darkgrid",palette="husl")
ax = sns.countplot(x="ChestPain", hue="Target", data=df) 
ax.set_title("Percentage of People with chest pain having heart disease")
ax.set(xlabel="ChestPain",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
total = len(df['Target'])
sns.set(style="darkgrid",palette="bright")
ax = sns.countplot(x="Fast_bld_sugar", hue="Target", data=df) 
ax.set_title("Percentage of People with high to low blood sugar having heart disease")
ax.set(xlabel="Fast_bld_sugar",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
total = len(df['Target'])
sns.set(style="darkgrid",palette="colorblind")
ax = sns.countplot(x="Rest_Ecg", hue="Target", data=df) 
ax.set_title("Percentage of People with normal to high ecg rates having heart disease")
ax.set(xlabel="Rest_Ecg",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
total = len(df['Target'])
sns.set(style="darkgrid",palette="PiYG")
ax = sns.countplot(x="Slope", hue="Target", data=df) 
ax.set_title("Percentage of People with slope level having heart disease")
ax.set(xlabel="Slope",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
total = len(df['Target'])
sns.set(style="darkgrid",palette="RdGy")
ax = sns.countplot(x="Thalassemia", hue="Target", data=df) 
ax.set_title("Percentage of People with fixed, normal, reversable thalassemia rates having heart disease")
ax.set(xlabel="Thalassemia",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
total = len(df['Sex'])
ax = sns.countplot(y="Sex", data=df, palette="Blues")
ax.set_title('Percentage of Male and Female')
plt.xlabel('No of people')

total = len(df['Sex'])
for p in ax.patches:
        percentage = '{:.1f}%'.format(100 * p.get_width()/total)
        x = p.get_x() + p.get_width() + 0.02
        y = p.get_y() + p.get_height()/2
        ax.annotate(percentage, (x, y))

plt.show()<jupyter_output><empty_output><jupyter_text>Percentage of Female Patients: 22.6%Percentage of Male Patients: 77.4%<jupyter_code>total = len(df['Target'])
sns.set(style="darkgrid",palette="mako_r")
ax = sns.countplot(x="Sex", hue="Target", data=df) 
ax.set_title("Percentage of People having heart disease according to gender")
ax.set(xlabel="Sex",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
df.sample(1)
total = len(df['Target'])
sns.set(style="darkgrid",palette="PRGn")
ax = sns.countplot(x="Ex_Angina", hue="Target", data=df) 
ax.set_title("Percentage of People with Angina having heart disease")
ax.set(xlabel="Ex_Angina",ylabel="Target")
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x()+p.get_width()/2.,
            height + 3,
            '{:.1%}'.format(height/total),
            ha="center") 
pd.crosstab(df.Age,df.Target).plot(kind="bar",figsize=(20,6),cmap='copper')
plt.title('Heart Disease Frequency for Ages')
plt.xlabel('Age')
plt.ylabel('Frequency')
plt.savefig('heartDiseaseAndAges.png')
plt.show()<jupyter_output><empty_output><jupyter_text>Univariant Analysis<jupyter_code>df.info()
df.sample(1)
df['Rest_bp']=df['Rest_bp'].astype(int)
df['Cholestrol']=df['Cholestrol'].astype(int)
df['St_Depr']=df['St_Depr'].astype(float)
df['Max_Rt']=df['Max_Rt'].astype(int)
sns.distplot(df['Age'],color="sienna")
sns.distplot(df['Rest_bp'],color="cadetblue")
sns.distplot(df['Cholestrol'],color="black")
sns.distplot(df['St_Depr'],color="darkred")
sns.distplot(df['Max_Rt'],color="coral")
df.sample(1)
df.info()
df = pd.concat([df, pd.get_dummies(df['Sex'])], axis=1)
df.drop('Sex',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['ChestPain'])], axis=1)
df.drop('ChestPain',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['Fast_bld_sugar'])], axis=1)
df.drop('Fast_bld_sugar',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['Rest_Ecg'])], axis=1)
df.drop('Rest_Ecg',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['Ex_Angina'])], axis=1)
df.drop('Ex_Angina',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['Slope'])], axis=1)
df.drop('Slope',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['Thalassemia'])], axis=1)
df.drop('Thalassemia',axis=1,inplace=True)
df = pd.concat([df, pd.get_dummies(df['Colored_Vessels'])], axis=1)
df.drop('Colored_Vessels',axis=1,inplace=True)
df.columns
df.shape
df.sample(3)
from sklearn.model_selection import train_test_split, KFold, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import confusion_matrix
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import SMOTE<jupyter_output>Using TensorFlow backend.<jupyter_text>Without using smote<jupyter_code>X = df.drop('Target',axis=1)
Y = df['Target']
# splitting data in train and test
X_train, X_valid, y_train, y_valid = train_test_split(X, Y, test_size = 0.30, random_state = 10)
sc= StandardScaler()
X_train = sc.fit_transform(X_train)
X_train=pd.DataFrame(X_train,columns=X.columns)
X_valid=sc.transform(X_valid)
# visualizing the results

from yellowbrick.target import ClassBalance
visualizer = ClassBalance(labels=[0, 1])

visualizer.fit(y_train,y_valid)
visualizer.ax.set_xlabel("Classes")
visualizer.ax.set_ylabel("Amount of Occurrences of Class")
visualizer.show()<jupyter_output><empty_output><jupyter_text>SVM<jupyter_code>from sklearn.metrics import confusion_matrix , classification_report
from sklearn.metrics import roc_curve , auc
from sklearn.svm import SVC
svm_classifier= SVC(probability=True, kernel='rbf')
svm_classifier.fit(X_train,y_train)
# Predict
y_pred_svm= svm_classifier.predict(X_valid)

#Classification Report
print(classification_report(y_valid,y_pred_svm))
mat_svm = confusion_matrix(y_valid, y_pred_svm, labels = [1,0])
sns.heatmap(mat_svm.T,  annot=True, fmt='d', cbar=False,
          xticklabels=['Yes','No'],
          yticklabels=['Yes','No'] )
plt.xlabel('true label')
plt.ylabel('predicted label')
print(mat_svm)
TP=mat_svm[0,0]
FN=mat_svm[0,1]
FP=mat_svm[1,0]
TN=mat_svm[1,1]
Recall=TP/(TP+FN)
print("Recall: ",Recall)
Precision=TP/(TP+FP)
print("Precision: ",Precision)
FM=(2*Recall*Precision)/(Recall+Precision)
print("F-Measure: ",FM)<jupyter_output>Recall:  0.9281437125748503
Precision:  0.8611111111111112
F-Measure:  0.8933717579250722<jupyter_text>The receiver operating characteristic (ROC) - SVM<jupyter_code>y_pred_svm_proba=svm_classifier.predict_proba(X_valid)[:,1]
fpr_svm, tpr_svm, _svm = roc_curve(y_valid, y_pred_svm_proba)
roc_auc=auc(fpr_svm,tpr_svm)

#Now Draw ROC using fpr , tpr
plt.plot([0, 1], [0, 1], 'k--',label='Random')

plt.plot(fpr_svm,tpr_svm,label='ROC curve (area = %0.2f)' %roc_auc)

plt.xlabel('False positive rate')
plt.ylabel('True positive rate')

plt.title('SVM ROC curve')
plt.legend(loc='best')
svm_classifier.score(X_valid,y_valid)
svm_classifier.score(X_train,y_train)
from yellowbrick.classifier import ClassificationReport
svccr = ClassificationReport(SVC(probability=True))
svccr.fit(X_train, y_train)
svccr.score(X_valid, y_valid)
svccr.show()<jupyter_output><empty_output><jupyter_text>Logistic Regression<jupyter_code>from sklearn.linear_model import LogisticRegression

# initiating the classifier and training the model

classifier = LogisticRegression(max_iter=1000)
classifier.fit(X_train, y_train)
# Predicting the test set results and calculating the accuracy

y_pred_logit = classifier.predict(X_valid)
classifier.score(X_valid, y_valid)
classifier.score(X_train, y_train)
# confusion matrix

from sklearn.metrics import confusion_matrix

matrix_logit = confusion_matrix(y_valid, y_pred_logit)
print(matrix_logit)
# Classification Report

from sklearn.metrics import classification_report
print(classification_report(y_valid, y_pred_logit))
sns.heatmap(matrix_logit,  annot=True, fmt='d', cbar=False,
          xticklabels=['Yes','No'],
          yticklabels=['Yes','No'] )
plt.xlabel('true label')
plt.ylabel('predicted label')
# Compute precision, recall, F-measure and support

TP=matrix_logit[0,0]
FN=matrix_logit[0,1]
FP=matrix_logit[1,0]
TN=matrix_logit[1,1]

Precision=TP/(TP+FP)
print("Precision: ",Precision)

Recall=TP/(TP+FN)
print("Recall: ",Recall)

FM=(2*Recall*Precision)/(Recall+Precision)
print("F-Measure: ",FM)
logit_roc_auc=classifier.predict_proba(X_valid)[:,1]
fpr,tpr,threshold=roc_curve(y_valid,logit_roc_auc)
roc_auc=auc(fpr,tpr)
plt.figure()

# ROC
plt.plot(fpr,tpr,'g',label='Logistic Regression (AUC = %0.2f)'% roc_auc)

# random FPR and TPR
plt.plot([0,1],[0,1],'r--')

# title and label
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Logistic Regression-Receiver operating characteristic')
plt.legend(loc='lower right')
plt.show()

print('The AUC:', auc(fpr,tpr))
from yellowbrick.classifier import ClassificationReport
lrcr = ClassificationReport(LogisticRegression(max_iter=1000))
lrcr.fit(X_train, y_train)
lrcr.score(X_valid, y_valid)
lrcr.show()<jupyter_output><empty_output><jupyter_text>K-Nearest Neighbours (KNN)<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
classifier = KNeighborsClassifier(n_neighbors = 10, metric = 'minkowski', p = 2)
classifier.fit(X_train, y_train)
from sklearn import metrics
y_pred_knn = classifier.predict(X_valid)
classifier.score(X_valid, y_valid)
classifier.score(X_train, y_train)
from sklearn.metrics import classification_report, confusion_matrix
mat_knn=confusion_matrix(y_valid, y_pred_knn)
print(mat_knn)
print(classification_report(y_valid, y_pred_knn))
sns.heatmap(mat_knn,  annot=True, fmt='d', cbar=False,
          xticklabels=['Yes','No'],
          yticklabels=['Yes','No'] )
plt.xlabel('true label')
plt.ylabel('predicted label')
TP=mat_knn[0,0]
FN=mat_knn[0,1]
FP=mat_knn[1,0]
TN=mat_knn[1,1]

knn_precision=TP/(TP+FP)
print("KNN Precision: ",knn_precision)

knn_recall=TP/(TP+FN)
print("KNN Recall: ",knn_recall)

knn_FM=(2*knn_recall*knn_precision)/(knn_recall+knn_precision)
print("KNN F-Measure: ",knn_FM)
##computing fpr and tpr we plot tpr vs fpr

knn_roc_auc=classifier.predict_proba(X_valid)[:,1]
fpr,tpr,threshold_smote=roc_curve(y_valid,knn_roc_auc)
roc_auc=auc(fpr,tpr)
plt.figure()

# ROC
plt.plot(fpr,tpr,'blue',label='KNN (AUC = %0.2f)'% roc_auc)

# random FPR and TPR
plt.plot([0,1],[0,1],'r--')

# title and label
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic')
plt.legend(loc='lower right')
plt.show()

print('The AUC (smote):',auc(fpr,tpr))
from yellowbrick.classifier import ClassificationReport
knncr = ClassificationReport(KNeighborsClassifier(n_neighbors = 10, metric = 'minkowski', p = 2))
knncr.fit(X_train, y_train)
knncr.score(X_valid, y_valid)
knncr.show()<jupyter_output><empty_output><jupyter_text>The above models SVM, Logistic and KNN have been observed and from the metrics used for classification i.e. the confusion matrix, accuracy score, precision, recall, f-measure and the roc-auc curve we can conclude that SVM works better than the other two models.  Using SMOTE Random Forest<jupyter_code>df1=df.drop(['Target'],axis=1)
array = df1.values 
arrayt=df['Target'].values
X = array
Y = arrayt
# splitting data in train and test
seed=600
X_train, X_valid, y_train, y_valid = train_test_split(X, Y, test_size = 0.30, random_state = seed)
from yellowbrick.target import ClassBalance
visualizer = ClassBalance(labels=[0, 1])

visualizer.fit(y_train,y_valid)
visualizer.ax.set_xlabel("Classes")
visualizer.ax.set_ylabel("Amount of Occurrences of Class")
visualizer.show()<jupyter_output><empty_output><jupyter_text>Synthetic Minority Oversampling Technique [SMOTE]<jupyter_code>from numpy import set_printoptions
from sklearn.metrics import confusion_matrix, classification_report
smt = SMOTE(random_state=seed)
X_train_SMOTE, Y_train_SMOTE = smt.fit_sample(X_train, y_train.ravel()) 
print(X_train_SMOTE.shape)
print(Y_train_SMOTE.shape)
set_printoptions(precision=3)
print('\n Oversampled input: \n %s' % X_train_SMOTE[0:5,:])
print("After OverSampling, counts of label '1': {}".format(sum(Y_train_SMOTE == 1))) 
print("After OverSampling, counts of label '0': {}".format(sum(Y_train_SMOTE == 0)))
from sklearn.model_selection import GridSearchCV
# Create the parameter grid based on the results of random search 
param_grid = {
    'max_features': [2, 3, 4, 5],
    'n_estimators': [200, 300, 400, 500]
}
# Create a based model
rf = RandomForestClassifier()
# Instantiate the grid search model
grid_search = GridSearchCV(estimator = rf, param_grid = param_grid,
                           cv = 3, n_jobs = -1, verbose = 2)
grid_result= grid_search.fit(X_train_SMOTE, Y_train_SMOTE)
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
# Applying Pipeline to random forest for 300 trees
from sklearn.pipeline import Pipeline 
num_trees=300
max_features=4
estimator=[]                                           
estimator.append(('standardize',StandardScaler()))
estimator.append(('RF',RandomForestClassifier(
    n_estimators=num_trees,max_features=max_features))) 
model=Pipeline(estimator)      
kfold=KFold(n_splits=10,random_state=seed)                 
result1=cross_val_score(model,X_train_SMOTE,Y_train_SMOTE,cv=kfold)  
print(result1.mean()*100.0) 
smt1 = SMOTE(random_state=seed)
X_train_SMOT1, Y_train_SMOT1 = smt1.fit_sample(X_train, y_train) 
estimator1=[]                                           
estimator1.append(('standardize',StandardScaler()))
estimator1.append(('RF',RandomForestClassifier(
    n_estimators=num_trees,max_features=max_features))) 
model1=Pipeline(estimator1) 
model1.fit(X_train_SMOT1, Y_train_SMOT1)
result = model1.score(X_valid, y_valid)
print((result)*100.0)
predictions = model1.predict(X_valid)
# print classification report 
print(classification_report(y_valid, predictions))
from sklearn.metrics import classification_report, confusion_matrix
classifier = RandomForestClassifier(n_estimators=num_trees,max_features=max_features)
classifier.fit(X_train, y_train)
y_pred_rf = classifier.predict(X_valid)
mat_rf=confusion_matrix(y_valid, y_pred_rf)
print(mat_rf)
# Compute precision, recall, F-measure and support

TP=mat_rf[0,0]
FN=mat_rf[0,1]
FP=mat_rf[1,0]
TN=mat_rf[1,1]

Precision=TP/(TP+FP)
print("Precision: ",Precision)

Recall=TP/(TP+FN)
print("Recall: ",Recall)

FM=(2*Recall*Precision)/(Recall+Precision)
print("F-Measure: ",FM)
sns.heatmap(mat_rf,  annot=True, fmt='d', cbar=False,
          xticklabels=['Yes','No'],
          yticklabels=['Yes','No'] )
plt.xlabel('true label')
plt.ylabel('predicted label')

rf_roc_auc_smote=classifier.predict_proba(X_valid)[:,1]
fpr,tpr,threshold_smote=roc_curve(y_valid,rf_roc_auc_smote)
roc_auc_smote=auc(fpr,tpr)
plt.figure()

# ROC
plt.plot(fpr,tpr,'blue',label='Random Forest (AUC = %0.2f)'% roc_auc_smote)

# random FPR and TPR
plt.plot([0,1],[0,1],'r--')

# title and label
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic')
plt.legend(loc='lower right')
plt.show()

print('The AUC (smote):',auc(fpr,tpr))
from yellowbrick.classifier import ClassificationReport
rfcr = ClassificationReport(classifier)
rfcr.fit(X_train, y_train)
rfcr.score(X_valid, y_valid)
rfcr.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14595496246872394, 'fraction_numerical': 0.052384923944556974, 'mean_word_length': 4.5304195036239845, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '112906', 'license': 'W3C', 'n_tokens_mistral': 10370, 'n_tokens_neox': 9350, 'n_words': 2180, 'path': 'Heart Disease Combined.ipynb', 'repo_name': 'Sgh23/Heart-Disease-Prediction', 'size': '25179'}