<jupyter_start><jupyter_text>OVERVIEW1. Announcements2. Sample code for data imbalance3. Work on a regression model 4. Learn about PDP and SHAP plots5. Talk about project module and SC<jupyter_code>%%capture
import sys

# If you're on Colab:
if 'google.colab' in sys.modules:
    DATA_PATH = 'https://raw.githubusercontent.com/LambdaSchool/DS-Unit-2-Applied-Modeling/master/data/'
    !pip install category_encoders==2.*
    !pip install pdpbox
    !pip install shap

# If you're working locally:
else:
    DATA_PATH = '../data/'<jupyter_output><empty_output><jupyter_text>Lambda School Data Science*Unit 2, Sprint 3, Module 4*--- Model Interpretation- Visualize and interpret **partial dependence plots**- Explain individual predictions with **shapley value plots** I. Wrangle Data<jupyter_code>import pandas as pd
def wrangle(filepath):
  df = pd.read_csv(filepath,skipinitialspace=True, parse_dates=['issue_d'], index_col = 'issue_d').sort_index()

  # taking int_rate as our target variable, remove the percentage symbol and convert to float
  # Use the strip function or replace function
  df['int_rate'] = df['int_rate'].str.strip('%').astype(float)

  # Consider only a subset where the term is 36 months
  df = df[df['term'] == '36 months']
  df.drop(columns = 'term', inplace = True)

  # Use regex to consolidate values in a feature
  keywords = ['business', 'consolidation|credit', 'home', 'car|vacation|medical|moving']
  for col in keywords:
    df[col] = (df['title']
              .fillna('')
              .str.lower()
              .str.contains(col,  regex=True)
              .astype(int))
    
  # Subset of the data / columns to retain
  keep_cols = keywords + ['annual_inc', 'fico_range_high', 
                          'funded_amnt', 'int_rate']
  df = df[keep_cols]
  return df

df = wrangle(DATA_PATH+'lending-club/lending-club-subset.csv')
df.head()<jupyter_output><empty_output><jupyter_text>columns with same values but different typecase.<jupyter_code>df.tail()<jupyter_output><empty_output><jupyter_text>II. Split Data<jupyter_code>target = 'int_rate'
y = df[target]
X = df.drop(columns = target)
X.shape
# Split data into training, validation and test in sequence
#X_train, y_train = 
X_train, y_train = X[:-20000], y[:-20000]
X_val, y_val = X[-20000:-10000], y[-20000:-10000]
X_test, y_test = X.tail(10000),y.tail(10000)
#X_test,y_test = X[-10000:], y[-10000:]
X_train<jupyter_output><empty_output><jupyter_text>III. Establish Baseline<jupyter_code>from sklearn.metrics import mean_absolute_error, r2_score
y_bl_pred = [y_train.mean()] * len(y_train)
print('Mean interest rate:', y_train.mean())
print('Baseline MAE:', mean_absolute_error(y_train,y_bl_pred))<jupyter_output>Mean interest rate: 11.559685527011519
Baseline MAE: 2.8988644656906475<jupyter_text>IV. Build Model<jupyter_code>from sklearn.linear_model import Ridge
from sklearn.ensemble import RandomForestRegressor
from xgboost import XGBRegressor
model_r = Ridge()
model_rfr = RandomForestRegressor()
model_xgbr = XGBRegressor()
model_r.fit(X_train,y_train)
model_rfr.fit(X_train,y_train)
model_xgbr.fit(X_train,y_train)<jupyter_output>[23:48:44] WARNING: /workspace/src/objective/regression_obj.cu:152: reg:linear is now deprecated in favor of reg:squarederror.<jupyter_text>V. Check Metrics<jupyter_code>def check_metrics(model):
  print(model)
  print('TRaining MAE', mean_absolute_error(y_train,model.predict(X_train)))
  print('Validation MAE', mean_absolute_error(y_val,model.predict(X_val)))
  print('Validation R2', model.score(X_val,y_val))
  #print('Validation R2', model.r2_score(model.predict(X_val),y_val))

model = [model_r,model_rfr, model_xgbr]

for m in model:
  check_metrics(m)<jupyter_output>Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)
TRaining MAE 2.3998616385044973
Validation MAE 2.6447705604356764
Validation R2 0.22255361069312085
RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',
                      max_depth=None, max_features='auto', max_leaf_nodes=None,
                      max_samples=None, min_impurity_decrease=0.0,
                      min_impurity_split=None, min_samples_leaf=1,
                      min_samples_split=2, min_weight_fraction_leaf=0.0,
                      n_estimators=100, n_jobs=None, oob_score=False,
                      random_state=None, verbose=0, warm_start=False)
TRaining MAE 1.1505192019260004
Validation MAE 2.774860269960257
Validation R2 0.13910566172041805
XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=1,
             colsample_bynode=1, colsample_bytree=1, gamma=0,
             importance_type='gain', lear[...]<jupyter_text>VI. Communicate Results Partial Dependence PlotsOne Feature<jupyter_code>from pdpbox.pdp import pdp_isolate, pdp_plot, pdp_interact, pdp_interact_plot
X_val.columns
isolate = pdp_isolate(model_xgbr,
                      dataset = X_val,
                      model_features = X_val.columns,
                      feature = 'business')

pdp_plot(isolate,feature_name = 'business')<jupyter_output><empty_output><jupyter_text>Two features<jupyter_code>features = ['consolidation|credit','fico_range_high']
interact = pdp_interact(model_xgbr,
                        dataset = X_val,
                        model_features = X_val.columns,
                        features=features)

pdp_interact_plot(interact,plot_type='grid',feature_names = features)<jupyter_output><empty_output><jupyter_text>Shapley Plots- Show you the influence of features in *individual* predictions.<jupyter_code>import shap
samp = X_val.tail(1)
samp
explainer = shap.TreeExplainer(model_xgbr) # SHAP DOESNT NOT TAKE IN PIPELINES
shap_values = explainer.shap_values(samp)

shap.initjs()
shap.force_plot(base_value=explainer.expected_value,
                shap_values = shap_values, 
                features=samp)

explainer=shap.TreeExplainer(model_xgbr)
shap_values=explainer(X_val)
shap.plots.waterfall(shap_values[0])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1116801585990418, 'fraction_numerical': 0.03998017511977532, 'mean_word_length': 4.579723502304147, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '52089', 'license': 'MIT', 'n_tokens_mistral': 2301, 'n_tokens_neox': 2094, 'n_words': 410, 'path': 'DS29_GP_Unit2_Sprint3_Module4_Nivi.ipynb', 'repo_name': 'nicholashagemann/lambdalectures', 'size': '6053'}