<jupyter_start><jupyter_text>Comparison of bank loan risks’ classification models In this notebook we try to compare some machine learning models on a loan classification case.We load a dataset using Pandas library. Then we apply some algorithmsto choose the best one.<jupyter_code>import itertools
import wget

import numpy as np
import pandas as pd

from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn import svm
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import jaccard_score, f1_score, log_loss

import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
from matplotlib.ticker import NullFormatter
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Training dataset<jupyter_code>wget.download("https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv", "loan_train.csv")<jupyter_output>100% [..............................................................................] 23101 / 23101<jupyter_text>Load Data From CSV File<jupyter_code>df = pd.read_csv('Data/loan_train.csv')
df.head()
df.shape<jupyter_output><empty_output><jupyter_text>Convert to date time object<jupyter_code>df['due_date'] = pd.to_datetime(df['due_date'])
df['effective_date'] = pd.to_datetime(df['effective_date'])
df.head()<jupyter_output><empty_output><jupyter_text>Data visualization and pre-processing<jupyter_code>print("Classes ditribution : ")
df['loan_status'].value_counts()<jupyter_output>Classes ditribution :<jupyter_text>260 people have paid off the loan on time while 86 have gone into collection  Some visualization to better underestand the data:<jupyter_code>bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)
g = sns.FacetGrid(df, col="Gender", hue="loan_status", palette="Set1", col_wrap=2)
g.map(plt.hist, 'Principal', bins=bins, ec="k")

g.axes[-1].legend()
plt.show()
bins=np.linspace(df.age.min(), df.age.max(), 10)
g = sns.FacetGrid(df, col="Gender", hue="loan_status", palette="Set1", col_wrap=2)
g.map(plt.hist, 'age', bins=bins, ec="k")

g.axes[-1].legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Pre-processing:  Feature selection/extraction<jupyter_code>df['dayofweek'] = df['effective_date'].dt.dayofweek
bins=np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)
g = sns.FacetGrid(df, col="Gender", hue="loan_status", palette="Set1", col_wrap=2)
g.map(plt.hist, 'dayofweek', bins=bins, ec="k")
g.axes[-1].legend()
plt.show()<jupyter_output><empty_output><jupyter_text>We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4<jupyter_code>df['weekend']= df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)
df.head()<jupyter_output><empty_output><jupyter_text>Convert Categorical features to numerical values<jupyter_code>print("Classes ditribution by gender : ")
df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)<jupyter_output>Classes ditribution by gender :<jupyter_text>86 % of female pay there loans while only 73 % of males pay there loan We convert male to 0 and female to 1:<jupyter_code>df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)
df.head()<jupyter_output><empty_output><jupyter_text>One Hot Encoding<jupyter_code>print("Classes ditribution by education : ")
df.groupby(['education'])['loan_status'].value_counts(normalize=True)<jupyter_output>Classes ditribution by education :<jupyter_text>Feature before One Hot Encoding<jupyter_code>df[['Principal','terms','age','Gender','education']].head()<jupyter_output><empty_output><jupyter_text>One hot encoding technique to convert categorical variables to binary variables and append them to the feature Dataframe<jupyter_code>Feature = df[['Principal','terms','age','Gender','weekend']]
Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)
Feature.drop(['Master or Above'], axis = 1,inplace=True)
Feature.head()<jupyter_output><empty_output><jupyter_text>Feature selection We define the feature set X:<jupyter_code>X = Feature
X[0:5]<jupyter_output><empty_output><jupyter_text>What are our labels?<jupyter_code>y = df['loan_status'].values
y[0:5]<jupyter_output><empty_output><jupyter_text>Normalize Data  Data Standardization give data zero mean and unit variance :<jupyter_code>X = preprocessing.StandardScaler().fit(X).transform(X)
X[0:5]<jupyter_output><empty_output><jupyter_text>Classification  We will use the following algorithm:- K Nearest Neighbor(KNN)- Decision Tree- Support Vector Machine- Logistic Regression  K Nearest Neighbor(KNN)<jupyter_code># We split the X into train and test to find the best k
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)
print ('Train set:', X_train.shape,  y_train.shape)
print ('Test set:', X_test.shape,  y_test.shape)
# Modeling
k = 3
#Train Model and Predict  
kNN_model = KNeighborsClassifier(n_neighbors=k).fit(X_train,y_train)
kNN_model
# just for sanity chaeck
yhat = kNN_model.predict(X_test)
yhat[0:5]
# Best k
Ks=15
mean_acc=np.zeros((Ks-1))
for n in range(1,Ks):
    
    #Train Model and Predict  
    kNN_model = KNeighborsClassifier(n_neighbors=n).fit(X_train,y_train)
    yhat = kNN_model.predict(X_test)
    
    mean_acc[n-1]=np.mean(yhat==y_test)
    
plt.plot(mean_acc)
# Building the model again, using k=7
k = np.argmax(mean_acc)

#Train Model and Predict  
kNN_model = KNeighborsClassifier(n_neighbors=k).fit(X_train,y_train)
kNN_model<jupyter_output><empty_output><jupyter_text>Decision Tree<jupyter_code>DT_model = DecisionTreeClassifier(criterion="entropy", max_depth = 4)
DT_model.fit(X_train,y_train)
DT_model
yhat = DT_model.predict(X_test)
yhat<jupyter_output><empty_output><jupyter_text>Support Vector Machine<jupyter_code>SVM_model = svm.SVC()
SVM_model.fit(X_train, y_train) 
yhat = SVM_model.predict(X_test)
yhat<jupyter_output><empty_output><jupyter_text>Logistic Regression<jupyter_code>LR_model = LogisticRegression(C=0.01).fit(X_train,y_train)
LR_model
yhat = LR_model.predict(X_test)
yhat<jupyter_output><empty_output><jupyter_text>Model Evaluation using Test set<jupyter_code>Recap = {}<jupyter_output><empty_output><jupyter_text>First, we download and load the test set:<jupyter_code>wget.download("https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv", "loan_test.csv")<jupyter_output>100% [................................................................................] 3642 / 3642<jupyter_text>Load Test set for evaluation<jupyter_code>test_df = pd.read_csv('Data/loan_test.csv')
test_df.head()
## Preprocessing
test_df['due_date'] = pd.to_datetime(test_df['due_date'])
test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])
test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek
test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)
test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)
test_Feature = test_df[['Principal','terms','age','Gender','weekend']]
test_Feature = pd.concat([test_Feature,pd.get_dummies(test_df['education'])], axis=1)
test_Feature.drop(['Master or Above'], axis = 1,inplace=True)
test_X = preprocessing.StandardScaler().fit(test_Feature).transform(test_Feature)
test_X[0:5]
test_y = test_df['loan_status'].values
test_y[0:5]
knn_yhat = kNN_model.predict(test_X)

f1 = f1_score(test_y, knn_yhat, average='weighted')
jaccard = jaccard_score(test_y, knn_yhat, average = "micro", labels=['COLLECTION', 'PAIDOFF'])

Recap["K-Nearest Neighbors"]=[f1, jaccard, None]

print("KNN F1-score: %.2f" %  f1)
print("KNN Jaccard index: %.2f" % jaccard)
DT_yhat = DT_model.predict(test_X)

f1 = f1_score(test_y, DT_yhat, average='weighted')
jaccard = jaccard_score(test_y, DT_yhat, average = "micro", labels=['COLLECTION', 'PAIDOFF'])

Recap["Decision Tree"]=[f1, jaccard, None]

print("DT Jaccard index: %.2f" % jaccard)
print("DT F1-score: %.2f" % f1)
SVM_yhat = SVM_model.predict(test_X)

f1 = f1_score(test_y, SVM_yhat, average='weighted')
jaccard = jaccard_score(test_y, SVM_yhat, average = "micro", labels=['COLLECTION', 'PAIDOFF'])

Recap["Support Vector Machine"]=[f1, jaccard, None]

print("SVM F1-score: %.2f" % f1 )
print("SVM Jaccard index: %.2f" % jaccard)
LR_yhat = LR_model.predict(test_X)
LR_yhat_prob = LR_model.predict_proba(test_X)

f1 = f1_score(test_y, LR_yhat, average='weighted')
jaccard = jaccard_score(test_y, LR_yhat, average = "micro", labels=['COLLECTION', 'PAIDOFF'])
loss = log_loss(test_y, LR_yhat_prob)

Recap["Logistic Loss"]=[f1, jaccard, loss]

print("LR F1-score: %.2f" % f1 )
print("LR Jaccard index: %.2f" % jaccard)
print("LR LogLoss: %.2f" % loss)
Recap_matrix = pd.DataFrame(Recap, index = ["F1-score", "Jaccard Index", "Logistic regression"])

Recap_matrix
Recap_matrix.idxmax(axis = 1, skipna = True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1547107438016529, 'fraction_numerical': 0.01487603305785124, 'mean_word_length': 6.975395430579965, 'pattern_counts': {'":': 0, '<': 88, '<?xml version=': 0, '>': 90, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '462236', 'license': 'MIT', 'n_tokens_mistral': 3493, 'n_tokens_neox': 3308, 'n_words': 640, 'path': 'Supervised learning projects/Loan classifier/Loan_classifier_main.ipynb', 'repo_name': 'Oussama-KIASSI/Data_Science', 'size': '9075'}