<jupyter_start><jupyter_text>Machine Learning with Decision Trees and Random ForestsIn this project we will be exploring publicly available data from [LendingClub.com](www.lendingclub.com). Lending Club is a service that connects borrowers with people who have money (investors). We will try to create a model that will help predict if a person has a high probability of paying the loan back.<jupyter_code>import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>DataWe will use lending data from 2007-2010 and try to classify and predict whether or not the borrower paid back their loan in full. The data set contains the following features:* credit.policy: 1 if the customer meets the credit underwriting criteria of LendingClub.com, and 0 otherwise.* purpose: The purpose of the loan (takes values "credit_card", "debt_consolidation", "educational", "major_purchase", "small_business", and "all_other").* int.rate: The interest rate of the loan, as a proportion (a rate of 11% would be stored as 0.11). Borrowers judged by LendingClub.com to be more risky are assigned higher interest rates.* installment: The monthly installments owed by the borrower if the loan is funded.* log.annual.inc: The natural log of the self-reported annual income of the borrower.* dti: The debt-to-income ratio of the borrower (amount of debt divided by annual income).* fico: The FICO credit score of the borrower.* days.with.cr.line: The number of days the borrower has had a credit line.* revol.bal: The borrower's revolving balance (amount unpaid at the end of the credit card billing cycle).* revol.util: The borrower's revolving line utilization rate (the amount of the credit line used relative to total credit available).* inq.last.6mths: The borrower's number of inquiries by creditors in the last 6 months.* delinq.2yrs: The number of times the borrower had been 30+ days past due on a payment in the past 2 years.* pub.rec: The borrower's number of derogatory public records (bankruptcy filings, tax liens, or judgments).<jupyter_code>#Reading the data
loans = pd.read_csv('loan_data.csv')
loans.info()
loans.describe()
loans.head()<jupyter_output><empty_output><jupyter_text>Exploratory Data AnalysisUsing quick visualisations, we can explore the relationship between different variables in the dataset. Let's start with a dual histogram of the FICO score of the borrowers, depending on the credit policy (i.e. if a borrower met the underlying criteria).<jupyter_code>loans[loans['credit.policy']==1]['fico'].hist(bins=30,alpha=0.6,label='Credit.Policy=1')
loans[loans['credit.policy']==0]['fico'].hist(bins=30,alpha=0.6,label='Credit.Policy=0')
plt.legend()
plt.xlabel('FICO')<jupyter_output><empty_output><jupyter_text>Checking out a similar histogram, based on the 'not.fully.paid' column.<jupyter_code>loans[loans['not.fully.paid']==1]['fico'].hist(bins=30,alpha=0.6,label='not.fully.paid=1',color='red')
loans[loans['not.fully.paid']==0]['fico'].hist(bins=30,alpha=0.6,label='not.fully.paid=0')
plt.legend()
plt.xlabel('FICO')<jupyter_output><empty_output><jupyter_text>Let's visualize the counts of of loan purposes, based on whether a borrower fully paid the loan back or not.<jupyter_code>sns.set_style('whitegrid')
plt.figure(figsize=(9,5))
sns.countplot(loans['purpose'],hue=loans['not.fully.paid'])
plt.tight_layout()<jupyter_output>C:\Users\<NAME>\anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(<jupyter_text>Let's see the trend between FICO score and interest rate.<jupyter_code>sns.jointplot(x='fico',y='int.rate',data=loans)
plt.figure(figsize=(11,7))
sns.lmplot(y='int.rate',x='fico',data=loans,hue='credit.policy',
           col='not.fully.paid',palette='Set1')<jupyter_output><empty_output><jupyter_text>Model Building The _purpose_ column is categorical.That means we need to transform them using dummy variables so sklearn will be able to understand them.<jupyter_code>#Creating a list of all categorical features.
cat_feats = ['purpose']
final_data = pd.get_dummies(loans,columns=cat_feats,drop_first=True)<jupyter_output><empty_output><jupyter_text>Train Test Split<jupyter_code>final_data.columns
X = final_data.drop('not.fully.paid',axis=1)
y = final_data['not.fully.paid']
from sklearn.model_selection import train_test_split

X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.3)<jupyter_output><empty_output><jupyter_text>Training a Decision Tree Model<jupyter_code>from sklearn.tree import DecisionTreeClassifier
dtree= DecisionTreeClassifier()
dtree.fit(X_train,y_train)<jupyter_output><empty_output><jupyter_text>Predictions and Evaluation of Decision Tree<jupyter_code>predictions = dtree.predict(X_test)
from sklearn.metrics import classification_report,confusion_matrix
print(classification_report(y_test,predictions))
print(confusion_matrix(y_test,predictions))<jupyter_output>[[2018  396]
 [ 354  106]]<jupyter_text>Training the Random Forest model<jupyter_code>from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier()
rfc.fit(X_train,y_train)<jupyter_output><empty_output><jupyter_text>Predictions and Evaluation of Random Forest<jupyter_code>predictions = rfc.predict(X_test)
print(classification_report(y_test,predictions))
print(confusion_matrix(y_test,predictions))<jupyter_output>[[2398   16]
 [ 451    9]]
{'fraction_non_alphanumeric': 0.10298826040554962, 'fraction_numerical': 0.014763429384560654, 'mean_word_length': 7.1729651162790695, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '89077', 'license': 'MIT', 'n_tokens_mistral': 1867, 'n_tokens_neox': 1741, 'n_words': 528, 'path': 'Machine Learning with Decision Trees and Random Forests.ipynb', 'repo_name': 'episkey24/ML-Micro-Projects', 'size': '5632'}