<jupyter_start><jupyter_text>Combining All ML Models to Compare in ROC Graph<jupyter_code># Load dependencies for all models
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np

from keras.models import Sequential
from sklearn.preprocessing import StandardScaler
from keras.utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense

import matplotlib
%matplotlib inline  
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression

from sklearn.metrics import precision_recall_fscore_support
from sklearn.metrics import recall_score
from sklearn.metrics import classification_report
import pickle

from sklearn.metrics import confusion_matrix
import itertools

from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve
df = pd.read_csv(".\\credit_cards_dataset.csv")
df.head()
print("Original shape of the data: "+ str(df.shape))
features_names = df.columns
X = df.drop('default.payment.next.month', axis =1).values
y = df['default.payment.next.month'].values
print(X.shape)
print(y.shape)<jupyter_output>(30000, 24)
(30000,)<jupyter_text>Split my data into training and testing<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)<jupyter_output><empty_output><jupyter_text>Instantiate the random forest model with 200 trees<jupyter_code>rf = RandomForestClassifier(n_estimators=200)
rf.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Check feature importance<jupyter_code>sorted(zip(rf.feature_importances_, features_names), reverse=True) 
## plot the importances ##
import matplotlib.pyplot as plt
importances = rf.feature_importances_


indices = np.argsort(importances)[::-1]
plt.figure(figsize=(12,6))
plt.title("Feature importances by DecisionTreeClassifier")
plt.bar(range(len(indices)), importances[indices], color='lightblue',  align="center")
plt.step(range(len(indices)), np.cumsum(importances[indices]), where='mid', label='Cumulative')
plt.xticks(range(len(indices)), features_names[indices], rotation='vertical',fontsize=14)
plt.xlim([-1, len(indices)])
plt.show()<jupyter_output><empty_output><jupyter_text>Making my prediction and seeing how well my model predicted by checking recall, precision, F1 score and making a confusion matrix.Recall -tells us generally or overall how well our model predicted based on the total of how much it correctly predicted /correctly predicted + how many where actually right but predicted wrong.                    formula = TP/TP+FNPrecision -tells us or gives us true measure how well our model predicted it shows correctly predicted /correctly predicted + how many the model predicted to be positive but where false.                   formula = TP/TP+FPF1 score - gives us a mean of precision and recall, a sumarization of how well it did in respect to recall and precision.<jupyter_code>#Make my predictions
rf_y_prediction = rf.predict(X_test)
rf_roc_auc = roc_auc_score(y_test, rf.predict(X_test))
print(classification_report(y_test, rf_y_prediction,
                            target_names=["non_default", "default"]))
def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
        print("Normalized confusion matrix")
    else:
        print('Confusion matrix, without normalization')

    print(cm)

    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    fmt = '.2f' if normalize else 'd'
    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, format(cm[i, j], fmt),
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('True label')
    plt.xlabel('Predicted label')
# Compute confusion matrix
cnf_matrix = confusion_matrix(y_test, rf_y_prediction)
np.set_printoptions(precision=2)

# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['non_default', 'default'],
                      title='Confusion matrix, without normalization')

# Plot normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['non_default', 'default'], normalize=True,
                      title='Normalized confusion matrix')

plt.show()<jupyter_output>Confusion matrix, without normalization
[[6632  408]
 [1239  721]]
Normalized confusion matrix
[[0.94 0.06]
 [0.63 0.37]]<jupyter_text>Explanation of this confusion matrixIn our confusion matrix, the non-default classification has a total of 2,158 points and defualt clssification has a total of 7,742 points.It correctly identified 7239 points as default and 503 points as non-default.Non_default classification incorectly predicted 1480 points as default and correctly classified 678 points as Non defualt.  Launch Neural Network Model<jupyter_code>X_scaler = StandardScaler().fit(X_train)
X_train_scaled = X_scaler.transform(X_train)
X_test_scaled = X_scaler.transform(X_test)
y_train_categorical = to_categorical(y_train)
y_test_categorical = to_categorical(y_test)
nn_model = Sequential()
nn_model.add(Dense(units = 30, activation ='relu', input_dim=24))
nn_model.add(Dense(units = 20, activation ='relu')) #second hidden layer

nn_model.add(Dense(units = 2, activation = 'softmax'))
nn_model.summary()
#compile the model
nn_model.compile(optimizer = 'adam' ,
             loss = 'categorical_crossentropy',
             metrics =['accuracy'])
#train the model

nn_model.fit(X_train_scaled, y_train_categorical, epochs=100,shuffle = True,verbose =2)
model_loss, model_accuracy = nn_model.evaluate(X_test_scaled,y_test_categorical,verbose =2)
print( model_loss )
print (model_accuracy)
nn_y_prediction = nn_model.predict_classes(X_test)
nn_roc_auc = roc_auc_score(y_test, nn_model.predict_classes(X_test))
print(classification_report(y_test, nn_y_prediction,
                            target_names=['non_default', 'default']))
# Compute confusion matrix
cnf_matrix = confusion_matrix(y_test, nn_y_prediction)
np.set_printoptions(precision=2)

# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['non_default', 'default'],
                      title='Confusion matrix, without normalization')

# Plot normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['non_default', 'default'], normalize=True,
                      title='Normalized confusion matrix')

plt.show()<jupyter_output>Confusion matrix, without normalization
[[6697  343]
 [1825  135]]
Normalized confusion matrix
[[0.95 0.05]
 [0.93 0.07]]<jupyter_text>Logistic Regression Model<jupyter_code>#creating logistic regression model
classifier = LogisticRegression()
classifier
#fitting the model
classifier.fit(X_train, y_train) 
log_y_prediction = classifier.predict(X_test)
log_roc_auc = roc_auc_score(y_test, classifier.predict(X_test))
print(classification_report(y_test, log_y_prediction,
                            target_names=['non_default', 'default' ]))
cnf_matrix = confusion_matrix(y_test, log_y_prediction)
np.set_printoptions(precision=2)

# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['non_default', 'default'],
                      title='Confusion matrix, without normalization')

# Plot normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=['non_default', 'default'], normalize=True,
                      title='Normalized confusion matrix')

plt.show()
fpr, tpr, thresholds = roc_curve(y_test, classifier.predict_proba(X_test)[:,1])
rf_fpr, rf_tpr, rf_thresholds = roc_curve(y_test, rf.predict_proba(X_test)[:,1])
nn_fpr, nn_tpr, nn_thresholds = roc_curve(y_test, nn_model.predict_proba(X_test)[:,1])

plt.figure()

# Plot Logistic Regression ROC
plt.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % log_roc_auc)

# Plot Random Forest ROC
plt.plot(rf_fpr, rf_tpr, label='Random Forest (area = %0.2f)' % rf_roc_auc)

# Plot Decision Tree ROC
plt.plot(nn_fpr, nn_tpr, label='Neural Network (area = %0.2f)' % nn_roc_auc)

# Plot Base Rate ROC
plt.plot([0,1], [0,1],label='Base Rate' 'k--')

plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Graph')
plt.legend(loc="lower right")
plt.savefig('.\\ROC.png')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10742601898380792, 'fraction_numerical': 0.01675041876046901, 'mean_word_length': 5.080108621860149, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '128541', 'license': 'MIT', 'n_tokens_mistral': 3022, 'n_tokens_neox': 2847, 'n_words': 699, 'path': 'Model/5-0Combined Notebooks.ipynb', 'repo_name': 'skawns0724/KOSA-Big-Data_Vision', 'size': '8955'}