<jupyter_start><jupyter_text>Hypothesis Genereration<jupyter_code>### Data Visualization libraries
categorical_columns = ['Gender', 'Married', 'Dependents', 'Education', 'Self_Employed', 'Property_Area','Credit_History','Loan_Amount_Term']
fig,axes = plt.subplots(4,2,figsize=(12,15))
for idx,cat_col in enumerate(categorical_columns):
    row,col = idx//2,idx%2
    sns.countplot(x=cat_col,data=train,hue='Loan_Status',ax=axes[row,col])
plt.subplots_adjust(hspace=1)
train['Total_Income'] = train['ApplicantIncome'] + train['CoapplicantIncome']
bins = [0,2500,4000,6000,81000]
group = ['Very Low','Low','Average', 'High']
train['Total_Income_bin'] = pd.cut(train['Total_Income'],bins,labels=group)
Total_Income_bin = pd.crosstab(train['Total_Income_bin'],train['Loan_Status'])
Total_Income_bin.div(Total_Income_bin.sum(1).astype(float), axis=0).plot(kind="bar", stacked=True)
plt.xlabel('Total_Income')
P = plt.ylabel('Percentage')

bins = [0,100,200,700]
group = ['Low','Average','High']
train['LoanAmount_bin'] = pd.cut(train['LoanAmount'],bins,labels=group)
LoanAmount_bin = pd.crosstab(train['LoanAmount_bin'],train['Loan_Status'])
LoanAmount_bin.div(LoanAmount_bin.sum(1).astype(float), axis=0).plot(kind="bar", stacked=True)
plt.xlabel('LoanAmount')
P = plt.ylabel('Percentage')<jupyter_output><empty_output><jupyter_text>From the above graphs, Hypothesis generation- Men have higher rate of receiving a loan than women. - Married people have a higher shot at getting loan than unmarried people. - As the number of dependents increase the change of you getting a loan decreases. - Higher the Education Level i.e. graduate level higher is the rate of approval of loan. - People with jobs have an easier time getting a loan approved than self - employed people. - While semiurban has the highes approval rate, the rejection rate in all three of the categories is comparitively the same. - Applicants who have repayed their previous debts should have higher chances of loan approval. - Majority of the loans taken are for 360 months. - High income will have more chances of loan approval. - The chances of loan approval will be high when the loan amount is less.<jupyter_code>train= train.drop([ 'LoanAmount_bin', 'Total_Income_bin','Total_Income'], axis=1)
# replacing 3+ in Dependents variable with 3 for both train and test set
train['Dependents'].replace('3+', 3, inplace=True)
# replacing Y and N in Loan_Status variable with 1 and 0 respectively
train['Loan_Status'].replace('N', 0, inplace=True)
train['Loan_Status'].replace('Y', 1, inplace=True)
train.head()<jupyter_output><empty_output><jupyter_text>Data Pre-processing  Missing value imputation<jupyter_code># check for missing values
train.isnull().sum()
def isNaN(string):
    return str(string) != string
                   
# Replace categorical missing values in train and test with the mode

train['Gender'] = train['Gender'].replace(np.nan, train['Gender'].mode()[0], regex=True)
train['Married'] = train['Married'].replace(np.nan, train['Married'].mode()[0], regex=True)
train['Dependents'] = train['Dependents'].replace(np.nan, train['Dependents'].mode()[0], regex=True)
train['Self_Employed'] = train['Self_Employed'].replace(np.nan, train['Self_Employed'].mode()[0], regex=True)
train['Credit_History'] = train['Credit_History'].replace(np.nan, train['Credit_History'].mode()[0], regex=True)
train.isnull().sum()
# Replace continuous values with the median for train
median_loanAmount = train['LoanAmount'].median()
median_loanAmount

# Replace continuous values with the median for test
train['LoanAmount'] = train['LoanAmount'].replace(np.nan, median_loanAmount, regex=True)
train['LoanAmount'].fillna(median_loanAmount, inplace=True)

# Replace continuous values with the median for train
train['Loan_Amount_Term'].value_counts()
train['Loan_Amount_Term'].fillna(360, inplace=True)
train.isnull().sum()
print(train.apply(lambda x: len(x.unique())))<jupyter_output>Loan_ID              614
Gender                 2
Married                2
Dependents             4
Education              2
Self_Employed          2
ApplicantIncome      505
CoapplicantIncome    287
LoanAmount           203
Loan_Amount_Term      10
Credit_History         2
Property_Area          3
Loan_Status            2
dtype: int64<jupyter_text>Feature EngineeringBased on the domain knowledge, we can come up with new features that might affect the target variable. We will create the following three new features:-  Total Income - As discussed during bivariate analysis we will combine the Applicant Income and Coapplicant Income. If the total income is high, chances of loan approval might also be high. -  Equated Monthly Installment - EMI is the monthly amount to be paid by the applicant to repay the loan. Idea behind making this variable is that people who have high EMI’s might find it difficult to pay back the loan. We can calculate the EMI by taking the ratio of loan amount with respect to loan amount term. -  Balance Income - This is the income left after the EMI has been paid. Idea behind creating this variable is that if this value is high, the chances are high that a person will repay the loan and hence increasing the chances of loan approval.<jupyter_code>#calculate total income 
train['Total_Income']=train['ApplicantIncome'] + train['CoapplicantIncome']
# create EMI feature
train['EMI'] = train['LoanAmount'] / train['Loan_Amount_Term']
# create new "Balance Income" variable
train['Balance Income'] = train['Total_Income'] - (train['EMI']*1000) # Multiply with 1000 to make the units equal 
train.head()
# drop Loan_ID 
train = train.drop(['Loan_ID','ApplicantIncome','CoapplicantIncome','Loan_Amount_Term'], axis=1)
# adding dummies to the dataset
train = pd.get_dummies(train)
train.head()
# drop "Loan_Status" and assign it to target variable
X = train.drop('Loan_Status', 1)
y = train.Loan_Status<jupyter_output><empty_output><jupyter_text>Split dataset into test and train<jupyter_code># split the data into train and cross validation set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,random_state=42)<jupyter_output><empty_output><jupyter_text>We will build the following models in this section.- Logistic Regression- Decision Tree- Random Forest- Naive Baiyes - Support Vector Machine   Logistic Regression<jupyter_code>model = LogisticRegression()
model.fit(X_train, y_train)
# make prediction
predictions = model.predict(X_test)
#accuracy
score = model.score(X_test, y_test)
print(score*100)
cm = confusion_matrix(y_test, model.predict(X_test))

fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(cm)
ax.grid(False)
ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))
ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))
ax.set_ylim(1.5, -0.5)
for i in range(2):
    for j in range(2):
        ax.text(j, i, cm[i, j], ha='center', va='center', color='red')
plt.show()<jupyter_output><empty_output><jupyter_text>with K fold cross validation with 10 folds<jupyter_code>model = LogisticRegression(random_state=1)
accuracy = cross_val_score(model, X, y, scoring='accuracy', cv = 10)
print(accuracy)
#get the mean of each fold 
print("Accuracy of Model with Cross Validation is:",accuracy.mean() * 100)<jupyter_output>[0.79365079 0.82539683 0.75409836 0.75409836 0.78688525 0.78688525
 0.86885246 0.85245902 0.80327869 0.83606557]
Accuracy of Model with Cross Validation is: 80.6167056986729<jupyter_text>Decision Tree<jupyter_code># import library
from sklearn import tree
model = tree.DecisionTreeClassifier(random_state=1)
model.fit(X_train, y_train)
# make prediction
predictions = model.predict(X_test)
#accuracy
score = model.score(X_test, y_test)
print(score*100)
cm = confusion_matrix(y_test, model.predict(X_test))

fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(cm)
ax.grid(False)
ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))
ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))
ax.set_ylim(1.5, -0.5)
for i in range(2):
    for j in range(2):
        ax.text(j, i, cm[i, j], ha='center', va='center', color='red')
plt.show()<jupyter_output><empty_output><jupyter_text>with K fold cross validation with 10 folds<jupyter_code>model = tree.DecisionTreeClassifier()
accuracy = cross_val_score(model, X, y, scoring='accuracy', cv = 10)
print(accuracy)
#get the mean of each fold 
print("Accuracy of Model with Cross Validation is:",accuracy.mean() * 100)<jupyter_output>[0.68253968 0.73015873 0.73770492 0.72131148 0.6557377  0.70491803
 0.80327869 0.81967213 0.72131148 0.75409836]
Accuracy of Model with Cross Validation is: 73.30731199583657<jupyter_text>Random Forest<jupyter_code># import library
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(random_state=1)
model.fit(X_train, y_train)
# make prediction
predictions = model.predict(X_test)
#accuracy
score = model.score(X_test, y_test)
print(score*100)
cm = confusion_matrix(y_test, model.predict(X_test))

fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(cm)
ax.grid(False)
ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))
ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))
ax.set_ylim(1.5, -0.5)
for i in range(2):
    for j in range(2):
        ax.text(j, i, cm[i, j], ha='center', va='center', color='red')
plt.show()<jupyter_output><empty_output><jupyter_text>with K fold cross validation with 10 folds<jupyter_code>model = RandomForestClassifier(random_state=42, max_depth=10, n_estimators=10)
accuracy = cross_val_score(model, X, y, scoring='accuracy', cv = 10)
print(accuracy)
#get the mean of each fold 
print("Accuracy of Model with Cross Validation is:",accuracy.mean() * 100)<jupyter_output>[0.76190476 0.79365079 0.73770492 0.75409836 0.73770492 0.7704918
 0.81967213 0.7704918  0.78688525 0.80327869]
Accuracy of Model with Cross Validation is: 77.35883424408014<jupyter_text>Naive Bayes<jupyter_code># import library
from sklearn.naive_bayes import GaussianNB
model = GaussianNB()
model.fit(X_train, y_train)
# make prediction
predictions = model.predict(X_test)
#accuracy
score = model.score(X_test, y_test)
print(score*100)
cm = confusion_matrix(y_test, model.predict(X_test))

fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(cm)
ax.grid(False)
ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))
ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))
ax.set_ylim(1.5, -0.5)
for i in range(2):
    for j in range(2):
        ax.text(j, i, cm[i, j], ha='center', va='center', color='red')
plt.show()<jupyter_output><empty_output><jupyter_text>with K fold cross validation with 10 folds<jupyter_code>model = GaussianNB()
accuracy = cross_val_score(model, X, y, scoring='accuracy', cv = 10)
print(accuracy)
#get the mean of each fold 
print("Accuracy of Model with Cross Validation is:",accuracy.mean() * 100)<jupyter_output>[0.80952381 0.82539683 0.72131148 0.75409836 0.80327869 0.7704918
 0.83606557 0.81967213 0.75409836 0.86885246]
Accuracy of Model with Cross Validation is: 79.6278948737965<jupyter_text>Support Vector Machine<jupyter_code># import library
from sklearn import svm
model = svm.SVC()
model.fit(X_train, y_train)
# make prediction
predictions = model.predict(X_test)
#accuracy
score = model.score(X_test, y_test)
print(score*100)
cm = confusion_matrix(y_test, model.predict(X_test))

fig, ax = plt.subplots(figsize=(6, 6))
ax.imshow(cm)
ax.grid(False)
ax.xaxis.set(ticks=(0, 1), ticklabels=('Predicted 0s', 'Predicted 1s'))
ax.yaxis.set(ticks=(0, 1), ticklabels=('Actual 0s', 'Actual 1s'))
ax.set_ylim(1.5, -0.5)
for i in range(2):
    for j in range(2):
        ax.text(j, i, cm[i, j], ha='center', va='center', color='red')
plt.show()<jupyter_output><empty_output><jupyter_text>with K fold cross validation with 10 folds<jupyter_code>model = svm.SVC()
accuracy = cross_val_score(model, X, y, scoring='accuracy', cv = 10)
print(accuracy)
#get the mean of each fold 
print("Accuracy of Model with Cross Validation is:",accuracy.mean() * 100)<jupyter_output>[0.66666667 0.68253968 0.68852459 0.68852459 0.68852459 0.70491803
 0.68852459 0.70491803 0.67213115 0.68852459]
Accuracy of Model with Cross Validation is: 68.73796513140775
{'fraction_non_alphanumeric': 0.12836676217765042, 'fraction_numerical': 0.06238231682357757, 'mean_word_length': 6.008605851979346, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 55, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '497107', 'license': 'MIT', 'n_tokens_mistral': 4767, 'n_tokens_neox': 4204, 'n_words': 1122, 'path': 'LoanPrediction.ipynb', 'repo_name': 'nishitde/Loan-Prediction-System', 'size': '12215'}