<jupyter_start><jupyter_text>The 14 features are describedbelow:- slope_of_peak_exercise_st_segment (type: int): the slope of the peak exercise ST segment, an electrocardiography read out indicating quality of blood flow to the heart- thal (type: categorical): results of thallium stress test measuring blood flow to the heart, with possible values normal, fixed_defect, reversible_defect- resting_blood_pressure (type: int): resting blood pressure- chest_pain_type (type: int): chest pain type (4 values)- num_major_vessels (type: int): number of major vessels (0-3) colored by flourosopy- fasting_blood_sugar_gt_120_mg_per_dl (type: binary): fasting blood sugar > 120 mg/dl- resting_ekg_results (type: int): resting electrocardiographic results (values 0,1,2)- serum_cholesterol_mg_per_dl (type: int): serum cholestoral in mg/dl- oldpeak_eq_st_depression (type: float): oldpeak = ST depression induced by exercise relative to rest, a measure of abnormality in electrocardiograms- sex (type: binary): 0: female, 1: male- age (type: int): age in years- max_heart_rate_achieved (type: int): maximum heart rate achieved (beats per minute)- exercise_induced_angina (type: binary): exercise-induced chest pain (0: False, 1: True)- heart_disease_present (type: binary): 0: heart disease not present, 1: heart disease present NOTE: The index in data corresponds to patient ID<jupyter_code>print(atributes)
data.head()
# Counts for heart_disease_present in training data
data[13].value_counts()
data = (data.astype({0: 'float64', 1: 'category', 2: 'float64', 3: 'float64', 4: 'float64', 5: 'float64', 6: 'float64', 7: 'float64',
                    8: 'float64', 9: 'float64', 10: 'float64', 11: 'float64', 12: 'float64', 13: 'int64'}))
print(data.dtypes)

# Average for each attribute whether or not heart_disease_present
data.groupby(13).mean()
# Averages as separated by thalium stress test
data.groupby(1).mean()
category_col = data.select_dtypes(['category']).columns 
data[category_col] = data[category_col].apply(lambda x: x.cat.codes)
print(data.head())
thal = {0:'fixed_defect', 1:'normal', 2:'reversible_defect'}
data.groupby(13).mean()
X = data.loc[:, data.columns != 13]
y = np.array(data.loc[:, data.columns == 13]).reshape(180,)
print(X.head())
X.shape
sm = SMOTE(random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
columns = X_train.columns
sm_data_X, sm_data_y = sm.fit_sample(X_train, y_train)
sm_data_X = pd.DataFrame(data = sm_data_X, columns = columns )
sm_data_y= pd.DataFrame(sm_data_y) #IDK what you are trying to do here with the oversampling, i think you are trying to counter the uneven dist?

scl = StandardScaler()
scale = scl.fit(X_train)


print("length of oversampled data is ",len(sm_data_X))
print("Number of no heart disease in oversampled data",len(sm_data_y[sm_data_y[0]==0]))
print("Number of heart disease",len(sm_data_y[sm_data_y[0]==1]))
print("Proportion of no heart disease data in oversampled data is ",len(sm_data_y[sm_data_y[0]==0])/len(sm_data_X))
print("Proportion of heart disease data in oversampled data is ",len(sm_data_y[sm_data_y[0]==1])/len(sm_data_X))
#WITHOUT data Normalization
logreg = LogisticRegression(multi_class = 'multinomial', solver='saga', penalty='l1', max_iter=10000, C=1)

# fit the model with taining data
logreg.fit(X_train,y_train) #there was a major dataleak here! make sure you dont test using the same data u used to train
y_pred=logreg.predict(X_test)
y_pred_prob = logreg.predict_proba(X_test)
print(f'Accuracy of logistic regression classifier on test set: {logreg.score(X_test, y_test)}')
lloss = log_loss(y_test, y_pred_prob)
print(f'log loss: {lloss}')<jupyter_output>Accuracy of logistic regression classifier on test set: 0.8333333333333334
log loss: 0.3536654718058519<jupyter_text>LogisticRegression(multi_class = 'multinomial', solver='saga', penalty='l1', max_iter=10000)    Accuracy of logistic regression classifier on test set: 0.8333333333333334    log loss: 0.3536771729189414 LogisticRegression(multi_class = 'multinomial', solver='saga', penalty='l2', max_iter=10000)    Accuracy of logistic regression classifier on test set: 0.8055555555555556    log loss: 0.36027304512739483 LogisticRegression(multi_class = 'multinomial', solver='sag', penalty='l2', max_iter=10000)    Accuracy of logistic regression classifier on test set: 0.8333333333333334    log loss: 0.3585651405396726 LogisticRegression(multi_class = 'multinomial', solver='lbfgs', penalty='l2', max_iter=10000)    Accuracy of logistic regression classifier on test set: 0.8333333333333334    log loss: 0.42470468568912534 LogisticRegression(multi_class = 'multinomial', solver='newton-cg', penalty='l2', max_iter=10000)    Accuracy of logistic regression classifier on test set: 0.8333333333333334    log loss: 0.42600636188623087 LogisticRegression(multi_class = 'ovr', solver='liblinear', penalty='l2', max_iter=10000)    Accuracy of logistic regression classifier on test set: 0.8611111111111112    log loss: 0.3770715033919158<jupyter_code>#WITH data Normalization
logreg = LogisticRegression(multi_class = 'multinomial', solver='saga', penalty='l1', max_iter=10000, C=0.2)

# fit the model with data
logreg.fit(scl.transform(X_train),y_train)
y_pred=logreg.predict(scl.transform((X_test)))
y_pred_prob = logreg.predict_proba(scl.transform(X_test))
print(f'Accuracy of logistic regression classifier on test set: {logreg.score(scl.transform(X_test), y_test)}')
lloss = log_loss(y_test, y_pred_prob)
print(f'log loss: {lloss}')



#WITH data Normalization
logregCV = LogisticRegressionCV(multi_class = 'multinomial', solver='saga', penalty='l1', max_iter=10000, Cs =100)

# fit the model with data
logregCV.fit(scl.transform(X_train),y_train)
y_predCV=logregCV.predict(scl.transform((X_test)))
y_pred_probCV = logregCV.predict_proba(scl.transform(X_test))
print(f'Accuracy of logistic regression classifier on test set: {logregCV.score(scl.transform(X_test), y_test)}')
llossCV = log_loss(y_test, y_pred_probCV)
print(f'log loss: {llossCV}')
# The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier to not label a sample as positive if it is negative.
# The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples.
# The F-beta score can be interpreted as a weighted harmonic mean of the precision and recall, where an F-beta score reaches its best value at 1 and worst score at 0.
# The F-beta score weights the recall more than the precision by a factor of beta. beta = 1.0 means recall and precision are equally important.
# The support is the number of occurrences of each class in y_test.

confMatrix = confusion_matrix(y_test, y_pred)
print(confMatrix)

print(classification_report(y_test, y_pred))


logit_roc_auc = roc_auc_score(y_test, y_pred)
fpr, tpr, thresholds = roc_curve(y_test, y_pred_prob[:,1])
plt.figure()
plt.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % logit_roc_auc)
plt.plot([0, 1], [0, 1],'r--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic')
plt.legend(loc="lower right")
plt.show()

# Linear SVM without normalization
svclassifier = SVC(kernel='linear')  
svclassifier.fit(X_train, y_train)  
y_pred = svclassifier.predict(X_test)  
print(confusion_matrix(y_test,y_pred))  
print(classification_report(y_test,y_pred))  
# Linear SVM with normalization
svclassifier = SVC(kernel='linear')  
svclassifier.fit(scl.transform(X_train), y_train)  
y_pred = svclassifier.predict(scl.transform(X_test))  
print(confusion_matrix(y_test,y_pred))  
print(classification_report(y_test,y_pred)) 
degrees =np.arange(2,11)
print(degrees)

for deg in degrees:
    svclassifier = SVC(kernel='poly', degree = deg, gamma = 'scale')  
    svclassifier.fit(X_train, y_train) 
    y_pred = svclassifier.predict(X_test)  
    print("Polynomial Degree: ", deg)
    print(confusion_matrix(y_test,y_pred))  
    print(classification_report(y_test,y_pred))  
svclassifier = SVC(kernel='poly', degree=3, gamma = 'auto', probability = True)  
svclassifier.fit(X_train, y_train) 
y_pred = svclassifier.predict(X_test)  
y_pred_prob = svclassifier.predict_proba(X_test)
print(confusion_matrix(y_test,y_pred))  
print(classification_report(y_test,y_pred)) 

svclassifier = SVC(kernel='poly', degree=3, gamma = 'scale', probability = True)  
svclassifier.fit(X_train, y_train) 
y_pred = svclassifier.predict(X_test)  
y_pred_prob = svclassifier.predict_proba(X_test)
print(confusion_matrix(y_test,y_pred))  
print(classification_report(y_test,y_pred)) 
svclassifier = SVC(kernel='poly', degree=3, gamma = 'auto')  
svclassifier.fit(X_train, y_train) 
y_pred = svclassifier.predict(X_test)  
print(confusion_matrix(y_test,y_pred))  
print(classification_report(y_test,y_pred)) 
svclassifier = SVC(kernel='linear')  
rfe = RFECV(svclassifier)
rfe = rfe.fit(X_train, y_train)
print(rfe.support_)
print(rfe.ranking_)
print(rfe.n_features_)
print(rfe.grid_scores_)
# Plot number of features VS. cross-validation scores
plt.figure()
plt.xlabel("Number of features selected")
plt.ylabel("Cross validation score (nb of correct classifications)")
plt.plot(range(1, len(rfe.grid_scores_) + 1), rfe.grid_scores_)
plt.show()
for n, i in enumerate(rfe.support_):
    if i:
        print(atributes[n+1])
        
logregmask = [0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1]

print(f"\nWith LogReg Classififier:")
mainfeatures = []
for n, i in enumerate(logregmask):
    if not i:
        mainfeatures.append(n)
        continue;
    if not rfe.support_[n]:
        mainfeatures.append(n)
            
print(mainfeatures)
prim = [1,3,4,8,9,12,13]
data.head()
data6 = copy.deepcopy(data)
data6.drop(axis=1, labels=mainfeatures , inplace=True)
othfeat = [0,5,6]
data.head()
datan = [copy.deepcopy(data),copy.deepcopy(data),copy.deepcopy(data),copy.deepcopy(data),copy.deepcopy(data),copy.deepcopy(data),copy.deepcopy(data)]
templist = [0,5,6,[0,5],[0,6],[5,6],[0,5,6]]
fet = np.arange(0,13)

for n,itm in enumerate(templist):
    if (n == 0):
        mask = np.in1d(fet, [0]+prim, invert=True)
        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    elif n == 1:
        mask = np.in1d(fet, [5]+prim, invert=True)

        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    elif n == 2:
        mask = np.in1d(fet, [6]+prim, invert=True)

        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    elif n == 3:
        mask = np.in1d(fet, [0,5]+prim, invert=True)

        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    elif n == 4:
        mask = np.in1d(fet, [0,6]+prim, invert=True)

        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    elif n == 5:
        mask = np.in1d(fet, [5,6]+prim, invert=True)

        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    elif n == 6:
        mask = np.in1d(fet, [0,5,6]+prim, invert=True)

        datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
        
        
    #s = set(mainfeatures)
    #for i in [itm]:
    #    s.remove(i)
    #print(s)
    #mask = np.in1d(fet, s, invert=True)
    #print(fet[mask])
    #datan[n] = copy.deepcopy(data)
    #datan[n].drop(axis=1, labels=fet[mask] , inplace=True)
    

data.head()
datan.insert(0, data6)
datan = np.array(datan)
print(datan.shape)
#X, y, X_train, X_test, y_train, y_test = ( for i in range(6))
print(datan[5].head())
for i, d in enumerate(datan):
    X = d.loc[:, d.columns != 13]
    y = np.array(d.loc[:, d.columns == 13]).reshape(180,)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
    
    svclassifier = SVC(kernel='poly', degree=3, gamma = 'auto')  
    svclassifier.fit(X_train, y_train) 
    y_pred = svclassifier.predict(X_test)
    print(confusion_matrix(y_test,y_pred))  
    print(classification_report(y_test,y_pred)) 
    print(f'Accuracy of support vector classifier on test set: {svclassifier.score(X_test, y_test)}')<jupyter_output>0   1    3    4    6    8    9    12  13
0z64un  1.0   1  2.0  0.0  2.0  0.0  1.0  0.0   0
ryoo3j  2.0   1  3.0  0.0  0.0  1.6  0.0  0.0   0
yt1s1x  1.0   1  4.0  3.0  2.0  0.0  1.0  1.0   1
l2xjde  1.0   2  4.0  0.0  0.0  0.0  1.0  0.0   1
oyt4ek  3.0   2  1.0  0.0  2.0  4.2  1.0  0.0   0
[[19  5]
 [ 1 11]]
              precision    recall  f1-score   support

           0       0.95      0.79      0.86        24
           1       0.69      0.92      0.79        12

   micro avg       0.83      0.83      0.83        36
   macro avg       0.82      0.85      0.82        36
weighted avg       0.86      0.83      0.84        36

Accuracy of support vector classifier on test set: 0.8333333333333334
[[19  5]
 [ 0 12]]
              precision    recall  f1-score   support

           0       1.00      0.79      0.88        24
           1       0.71      1.00      0.83        12

   micro avg       0.86      0.86      0.86        36
   macro avg       0.85      0.90      0.86    [...]
{'fraction_non_alphanumeric': 0.12578378786734154, 'fraction_numerical': 0.05643272644859107, 'mean_word_length': 4.333601933924254, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '332460', 'license': 'MIT', 'n_tokens_mistral': 5307, 'n_tokens_neox': 4756, 'n_words': 1222, 'path': 'sandbox.ipynb', 'repo_name': 'Adigorla/CVDprediction', 'size': '13237'}