<jupyter_start><jupyter_text>Introduction  The intention of this notebook is to utilize tensorflow to build a neural network that helps to predict default likelihood, and to visualize some of the insights generated from the study. This kernel will evolve over time as I continue to add features and study the Lending Club data   Dependencies  Below the data and some external libraries are imported to begin the process<jupyter_code>#%matplotlib inline
import numpy as np
import pandas as pd
import itertools
from sklearn import preprocessing
import matplotlib.pyplot as plt
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import tensorflow as tf
from tensorflow.contrib.learn.python.learn import metric_spec
from tensorflow.contrib.learn.python.learn.estimators import _sklearn
from tensorflow.contrib.learn.python.learn.estimators import estimator
from tensorflow.contrib.learn.python.learn.estimators import model_fn
from tensorflow.python.framework import ops
from tensorflow.python.saved_model import loader
from tensorflow.python.saved_model import tag_constants
from tensorflow.python.util import compat
tf.logging.set_verbosity(tf.logging.FATAL) 
df = pd.read_csv("../input/loan.csv", low_memory=False)<jupyter_output><empty_output><jupyter_text>Creating the Target Label  From a prior notebook, I examined the 'loan_status' column. The cell below creates a column with binary value 0 for loans not in default, and binary value 1 for loans in default.<jupyter_code>df['Default_Binary'] = int(0)
for index, value in df.loan_status.iteritems():
    if value == 'Default':
        df.set_value(index,'Default_Binary',int(1))
    if value == 'Charged Off':
        df.set_value(index, 'Default_Binary',int(1))
    if value == 'Late (31-120 days)':
        df.set_value(index, 'Default_Binary',int(1))    
    if value == 'Late (16-30 days)':
        df.set_value(index, 'Default_Binary',int(1))
    if value == 'Does not meet the credit policy. Status:Charged Off':
        df.set_value(index, 'Default_Binary',int(1))<jupyter_output><empty_output><jupyter_text>Creating a category feature for "Loan Purpose"  Below I create a new column for loan purpose, and assign each type of loan purpose an integer value.<jupyter_code>df['Purpose_Cat'] = int(0) 
for index, value in df.purpose.iteritems():
    if value == 'debt_consolidation':
        df.set_value(index,'Purpose_Cat',int(1))
    if value == 'credit_card':
        df.set_value(index, 'Purpose_Cat',int(2))
    if value == 'home_improvement':
        df.set_value(index, 'Purpose_Cat',int(3))    
    if value == 'other':
        df.set_value(index, 'Purpose_Cat',int(4))    
    if value == 'major_purchase':
        df.set_value(index,'Purpose_Cat',int(5))
    if value == 'small_business':
        df.set_value(index, 'Purpose_Cat',int(6))
    if value == 'car':
        df.set_value(index, 'Purpose_Cat',int(7))    
    if value == 'medical':
        df.set_value(index, 'Purpose_Cat',int(8))   
    if value == 'moving':
        df.set_value(index, 'Purpose_Cat',int(9))    
    if value == 'vacation':
        df.set_value(index,'Purpose_Cat',int(10))
    if value == 'house':
        df.set_value(index, 'Purpose_Cat',int(11))
    if value == 'wedding':
        df.set_value(index, 'Purpose_Cat',int(12))    
    if value == 'renewable_energy':
        df.set_value(index, 'Purpose_Cat',int(13))     
    if value == 'educational':
        df.set_value(index, 'Purpose_Cat',int(14))<jupyter_output><empty_output><jupyter_text>Scaling Interest Rates  Below I scale the interest rate for each loan to a value between 0 and 1<jupyter_code>x = np.array(df.int_rate.values).reshape(-1,1) 
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
df['int_rate_scaled'] = pd.DataFrame(x_scaled)
print (df.int_rate_scaled[0:5])<jupyter_output><empty_output><jupyter_text>Scaling Loan Amount  Below I scale the amount funded for each loan to a value between 0 and 1<jupyter_code>x = np.array(df.funded_amnt.values).reshape(-1,1) 
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
df['funded_amnt_scaled'] = pd.DataFrame(x_scaled)
print (df.funded_amnt_scaled[0:5])<jupyter_output><empty_output><jupyter_text>Setting up the Neural Network  Below I split the data into a training, testing, and prediction set  After that, I assign the feature and target columns, and create the function that will be used to pass the data into the model<jupyter_code>training_set = df[0:500000] # Train on first 500k rows
testing_set = df[500001:849999] # Test on next 350k rows
prediction_set = df[850000:] # Predict on final 37k rows

COLUMNS = ['Purpose_Cat','funded_amnt_scaled','int_rate_scaled','Default_Binary']          
FEATURES = ['Purpose_Cat','funded_amnt_scaled','int_rate_scaled']
LABEL = 'Default_Binary'

def input_fn(data_set):
    feature_cols = {k: tf.constant(data_set[k].values) for k in FEATURES} 
    labels = tf.constant(data_set[LABEL].values)
    return feature_cols, labels<jupyter_output><empty_output><jupyter_text>Fitting The Model<jupyter_code>feature_cols = [tf.contrib.layers.real_valued_column(k)
              for k in FEATURES]
#config = tf.contrib.learn.RunConfig(keep_checkpoint_max=1) ######## DO NOT DELETE
regressor = tf.contrib.learn.DNNRegressor(
  feature_columns=feature_cols, hidden_units=[10, 20, 10], ) 
regressor.fit(input_fn=lambda: input_fn(training_set), steps=251)<jupyter_output><empty_output><jupyter_text>Evaluating the Model<jupyter_code># Score accuracy
ev = regressor.evaluate(input_fn=lambda: input_fn(testing_set), steps=10)
loss_score = ev["loss"]
print("Loss: {0:f}".format(loss_score))<jupyter_output><empty_output><jupyter_text>Predicting on new data<jupyter_code>y = regressor.predict(input_fn=lambda: input_fn(prediction_set))
predictions = list(itertools.islice(y, 37379))<jupyter_output><empty_output><jupyter_text>Visualize Predictions Relative To Interest Rates<jupyter_code>plt.plot(prediction_set.int_rate_scaled, predictions, 'ro')
plt.ylabel("Model Prediction Value")
plt.xlabel("Interest Rate of Loan (Scaled between 0-1)")
plt.show()<jupyter_output><empty_output><jupyter_text>Visualize Predictions Relative to Loan Size<jupyter_code>plt.plot(prediction_set.funded_amnt_scaled, predictions, 'ro')
plt.ylabel("Model Prediction Value")
plt.xlabel("Funded Amount of Loan (Scaled between 0-1)")
plt.show()<jupyter_output><empty_output><jupyter_text>Visualize Predictions Relative to Loan Purpose<jupyter_code>plt.plot(prediction_set.Purpose_Cat, predictions, 'ro')
plt.ylabel("Default Prediction Value")
plt.xlabel("Loan Purpose")
plt.title("DNN Regressor Predicting Default By Loan Purpose")
fig_size = plt.rcParams["figure.figsize"]
fig_size[0] = 8
fig_size[1] = 8
plt.rcParams["figure.figsize"] = fig_size
labels = ['Debt Consolidation', 'Credit Card', 'Home Improvement', 'Other',
         'Major Purchase', 'Small Business', 'Car', 'Medical',
         'Moving', 'Vacation', 'House', 'Wedding',
         'Renewable Energy']

plt.xticks([1,2,3,4,5,6,7,8,9,10,11,12,13,14], labels, rotation='vertical')

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11681268251981644, 'fraction_numerical': 0.017660965095257962, 'mean_word_length': 5.544131028207461, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '495179', 'license': 'MIT', 'n_tokens_mistral': 2544, 'n_tokens_neox': 2385, 'n_words': 572, 'path': 'downloaded_kernels/loan_data/kernel_152.ipynb', 'repo_name': 'josepablocam/common-code-extraction', 'size': '7191'}