<jupyter_start><jupyter_text>Description of the Dataset | Column Name                     | Description                                                                               || ------------------------------- |:-----------------------------------------------------------------------------------------:|  | SeriousDlqin2yrs                | In the dataset, this particular attribute indicates whether the borrower has experienced any past dues until 90 days in the previous 2 years.                                || Amount.Funded.By.Investors      | The amount (in dollars) loaned to the individual.                                         | | RevolvingUtilizationOfUnsecuredLines                   | This attribute indicates the credit card limits of the borrower after excluding any current loan debt and real estate.                                                                || Age                     | This attribute is self-explanatory. It indicates the borrower's age.                                               || NumberOfTime30-59DaysPastDueNotWorse                    | The number of this attribute indicates the number of times borrowers have paid their EMIs late but have paid them 30 days after the due date or 59 days before the due date.                                       || DebtRatio            | This is also a self-explanatory attribute, but we will try and understand it better with an example.                  || Monthly.Income                  | This attribute contains the value of the monthly income of borrowers.                                         || NumberOfOpenCreditLinesAndLoans                      |This attribute indicates how many times a borrower has paid their dues 90 days after the due date of their EMIs.                                                                                                                 || NumberRealEstateLoansOrLines               | This attribute indicates the number of loans the borrower holds for their real estate or the number of home loans a borrower has         |NumberOfTime60-89DaysPastDueNotWorse| This attribute indicates how many times borrowers have paid their EMIs late but paid them 60 days after their due date or 89 days before their due date.                                         || NumberOfDependents  |  This attribute is self-explanatory as well. It indicates the number of dependent family members the borrowers have. The dependent count is excluding the borrower.                |<jupyter_code>data2=pd.read_csv('credit-risk-modelling/data/cs-test.csv')

data2.head()
training_data = data.drop('Unnamed: 0', axis = 1,inplace=True)
# After droping the "Unnamed column"
data.head()
# For each column heading we replace "-" and convert the heading in lowercase 
cleancolumn = []
for i in range(len(data.columns)):
    cleancolumn.append(data.columns[i].replace('-', '').lower())
data.columns = cleancolumn
data.head()
data.isnull().sum()
data.shape

# Graphical representation of the missing values.
x = data.columns
y = data.isnull().sum()
sns.set()
sns.barplot(x,y)
ax = plt.gca()
for p in ax.patches:
    height = p.get_height()
    ax.text(p.get_x() + p.get_width()/2.,
            height + 2,
            int(height),
            fontsize=14, ha='center', va='bottom')
sns.set(font_scale=1.5)
ax.set_xlabel("Data Attributes")
ax.set_ylabel("count of missing records for each attribute")
plt.xticks(rotation=90)
plt.show()
# fill the missing value
training_data = data.fillna((data.mean()))
training_data.head()
training_data.isnull().sum()
# Finding the Correlation
# Check out the relation among different variables.
sns.set()
sns.set(font_scale=1.25)
sns.heatmap(training_data[training_data.columns[1:]].corr(),annot=True,fmt=".1f")
plt.show()
data.head()
from sklearn.ensemble import RandomForestClassifier
data.columns[1:]


# treating the nan values 
np.where(np.isnan(data))
data.fillna(data.mean(),inplace=True)
# Lets again check for the nan values.
np.where(np.isnan(data))
# no missing values
X = data.drop('seriousdlqin2yrs', axis=1)
y = data.seriousdlqin2yrs
features_label = training_data.columns[1:]
forest = RandomForestClassifier (n_estimators = 100, random_state=0, n_jobs = -1)
# we try to 100 estimator as i am training it on my lil CPU.So let's reduce the computation time.
forest.fit(X,y)
importances = forest.feature_importances_
indices = np. argsort(importances)[::-1]
for i in range(X.shape[1]):
    print ("%2d) %-*s %f" % (i + 1, 30, features_label[i],importances[indices[i]]))<jupyter_output>1) revolvingutilizationofunsecuredlines 0.191850
 2) age                            0.178629
 3) numberoftime3059dayspastduenotworse 0.145691
 4) debtratio                      0.127786
 5) monthlyincome                  0.093372
 6) numberofopencreditlinesandloans 0.088704
 7) numberoftimes90dayslate        0.050238
 8) numberrealestateloansorlines   0.045730
 9) numberoftime6089dayspastduenotworse 0.043924
10) numberofdependents             0.034076<jupyter_text>The result could be more better if you go form form estimator like try for '10000' . Whole process results better accuracy.<jupyter_code>plt.title('Feature Importances')
plt.bar(range(X.shape[1]),importances[indices], color="green", align="center")
plt.xticks(range(X.shape[1]),features_label, rotation=90)
plt.xlim([-1, X.shape[1]])
plt.show()<jupyter_output><empty_output><jupyter_text>Train and build baseline model.<jupyter_code>from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import AdaBoostClassifier, GradientBoostingClassifier, RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score
X = data.drop('seriousdlqin2yrs', axis=1)
y = data.seriousdlqin2yrs
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)
# Train & test on kNN
knMod = KNeighborsClassifier(n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2,
                             metric='minkowski', metric_params=None)
knMod.fit(X_train, y_train)
knMod.score(X_test, y_test)
test_labels=knMod.predict_proba(np.array(X_test.values))[:,1]
roc_auc_score(y_test,test_labels , average='macro', sample_weight=None)<jupyter_output><empty_output><jupyter_text>ROC_AUC score is very poor so let's train it with different model.  Logistric regression<jupyter_code>glmMod = LogisticRegression(penalty='l1', dual=False, tol=0.0001, C=1.0, fit_intercept=True,
                            intercept_scaling=1, class_weight=None, 
                            random_state=None, solver='liblinear', max_iter=100,
                            multi_class='ovr', verbose=2)


glmMod.fit(X_train, y_train)
glmMod.score(X_test, y_test)
test_labels=glmMod.predict_proba(np.array(X_test.values))[:,1]
roc_auc_score(y_test,test_labels , average='macro', sample_weight=None)<jupyter_output><empty_output><jupyter_text>Produce better result compare to the knn one .   AdaBoostClassifier<jupyter_code>adaMod = AdaBoostClassifier(base_estimator=None, n_estimators=200, learning_rate=1.0)
adaMod.fit(X_train, y_train)

adaMod.score(X_test, y_test)
test_labels=adaMod.predict_proba(np.array(X_test.values))[:,1]
roc_auc_score(y_test,test_labels , average='macro', sample_weight=None)<jupyter_output><empty_output><jupyter_text>Hurry... we got a better model.  Testing on Real Test Dataset<jupyter_code># Read Training dataset as well as drop the index column
test_data = pd.read_csv('credit-risk-modelling/data/cs-test.csv').drop('Unnamed: 0', axis = 1)
# For each column heading we replace "-" and convert the heading in lowercase 
cleancolumn = []
for i in range(len(test_data.columns)):
    cleancolumn.append(test_data.columns[i].replace('-', '').lower())
test_data.columns = cleancolumn
test_data.drop(['seriousdlqin2yrs'], axis=1, inplace=True)
test_data.fillna((training_data.median()), inplace=True)
test_data.head()
test_data.shape
# Lookout for null values.
test_data.isnull().sum()
# Look out for nan values.
np.where(np.isnan(test_data))

test_labels_adaMod = adaMod.predict_proba(np.array(test_data.values))[:,1]
print (len(test_labels_adaMod))


output = pd.DataFrame({'ID':test_data.index, 'probability':test_labels_adaMod})


output.head()
len(output[output['probability']>.5])<jupyter_output><empty_output><jupyter_text>How to improve model accuracy - We haven't used cross-validation techniques extensively in order to check the overfitting issue. For that we can use "  sklearn crossl val method"- We need to tune these hyperparameters in such a way that we can increase the accuracy of the classifier. There are various hyperparameter-tuning techniques that we need to use.  Save the model using pickle<jupyter_code>import pickle
filename = 'finalized_model.sav'
pickle.dump(adaMod, open(filename, 'wb'))
# load the pickled model.
loaded_model = pickle.load(open('finalized_model.sav', 'rb'))
result = loaded_model.predict_proba(np.array(test_data.values))
print(result)<jupyter_output>[[0.50352012 0.49647988]
 [0.50266341 0.49733659]
 [0.50520768 0.49479232]
 ...
 [0.50624527 0.49375473]
 [0.50329839 0.49670161]
 [0.50363721 0.49636279]]<jupyter_text>In the output list  you can see there are two probabilities : first one is probability of zero and second one is probability of one.<jupyter_code># lets produce our output in the form of binary.
result2=loaded_model.predict(test_data)
print(result2[:500])
test_data.head(5)
result1=loaded_model.predict([[32,3,2,600000,50000,2,3,3,1]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11120401337792642, 'fraction_numerical': 0.03459448160535117, 'mean_word_length': 3.3774016468435497, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 23, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '388010', 'license': 'MIT', 'n_tokens_mistral': 3144, 'n_tokens_neox': 2868, 'n_words': 770, 'path': 'LoanPrediction(_Predict_to_whom_financial_institutes_should_give_loans_or_credit_).ipynb', 'repo_name': 'AIVenture0/Model-Building', 'size': '9568'}