<jupyter_start><jupyter_text>Preliminaries: imports, start H2O, load data<jupyter_code>import sklearn
import pandas as pd
import numpy as np
import shap
import h2o
from h2o.automl import H2OAutoML
df = pd.read_csv('C:/Users/Karti/NEU/data/insurance.csv')
df.head()
h2o.init()
data_path = 'C:/Users/Karti/NEU/data/insurance.csv'
h2o_df = h2o.import_file(data_path)
splits = h2o_df.split_frame(ratios=[0.8],seed=1)
train = splits[0]
test = splits[1]
y = "charges" 
x = h2o_df.columns 
x.remove(y)<jupyter_output><empty_output><jupyter_text>Run h2o AutoML<jupyter_code>aml = H2OAutoML(max_runtime_secs=180, seed=1)
aml.train(x=x,y=y, training_frame=train)
lb = aml.leaderboard
lb.head()
# Get model ids for all models in the AutoML Leaderboard
model_ids = list(aml.leaderboard['model_id'].as_data_frame().iloc[:,0])
# Get the "All Models" Stacked Ensemble model
se = h2o.get_model([mid for mid in model_ids if "StackedEnsemble_AllModels" in mid][0])
# Get the Stacked Ensemble metalearner model
metalearner = h2o.get_model(se.metalearner()['name'])
pred = aml.predict(test)
pred.head()
lb = h2o.automl.get_leaderboard(aml, extra_columns = 'ALL')
lb<jupyter_output><empty_output><jupyter_text>View 'Black box' model<jupyter_code>#extract the best model in leaderboard list
bst_model = aml.leader
bst_model<jupyter_output>Model Details
=============
H2OStackedEnsembleEstimator :  Stacked Ensemble
Model Key:  StackedEnsemble_AllModels_AutoML_20200521_160928

No model summary for this model

ModelMetricsRegressionGLM: stackedensemble
** Reported on train data. **

MSE: 15666345.162742808
RMSE: 3958.0734155322093
MAE: 2205.8215304305054
RMSLE: 0.360771370206251
R^2: 0.8881586253169274
Mean Residual Deviance: 15666345.162742808
Null degrees of freedom: 1065
Residual degrees of freedom: 1057
Null deviance: 149321518899.49396
Residual deviance: 16700323943.483833
AIC: 20705.62597511269

ModelMetricsRegressionGLM: stackedensemble
** Reported on cross-validation data. **

MSE: 19517768.178981844
RMSE: 4417.89182517882
MAE: 2393.5148730881783
RMSLE: 0.4048608000034505
R^2: 0.8606634795029184
Mean Residual Deviance: 19517768.178981844
Null degrees of freedom: 1065
Residual degrees of freedom: 1053
Null deviance: 149600136035.9225
Residual deviance: 20805940878.794647
AIC: 20947.94391273476<jupyter_text>Use a decision tree surrogate to generate explanations of the "black box" model First bind the "black box" model predictions onto the training frame<jupyter_code>preds = bst_model.predict(h2o_df)
preds.columns = ['predicted_charges']
frame_yhat = h2o_df.cbind(preds)<jupyter_output>stackedensemble prediction progress: |████████████████████████████████████| 100%<jupyter_text>Train decision tree surrogate model<jupyter_code>from h2o.estimators.gbm import H2OGradientBoostingEstimator


yhat = 'predicted_charges'
model_id = 'dt_surrogate_mojo'

# train single tree surrogate model
surrogate = H2OGradientBoostingEstimator(ntrees=1,
                                         sample_rate=1,
                                         col_sample_rate=1,
                                         max_depth=3,
                                         seed=12345,
                                         model_id=model_id)

_ = surrogate.train(x=x, y=yhat, training_frame=frame_yhat)

# persist MOJO (compiled, representation of trained model)
# from which to generate plot of surrogate
mojo_path = surrogate.download_mojo(path='.')

print(surrogate)
print('Generated MOJO path:\n', mojo_path)<jupyter_output>gbm Model Build progress: |███████████████████████████████████████████████| 100%
Model Details
=============
H2OGradientBoostingEstimator :  Gradient Boosting Machine
Model Key:  dt_surrogate_mojo


Model Summary:<jupyter_text>Generate GraphViz representation of MOJO<jupyter_code>from h2o.backend import H2OLocalServer
# for calling external processes
import os
import re
import subprocess
from subprocess import CalledProcessError
import time



details = False # print more info on tree, details = True
title = 'Medical Charges Tree Surrogate'

hs = H2OLocalServer()
h2o_jar_path = hs._find_jar()
print('Discovered H2O jar path:\n', h2o_jar_path)

gv_file_name = model_id + '.gv'
gv_args = str('-cp ' + h2o_jar_path +
              ' hex.genmodel.tools.PrintMojo --tree 0 -i '
              + mojo_path + ' -o').split()
gv_args.insert(0, 'java')
gv_args.append(gv_file_name)

if details:
    gv_args.append('--detail')

if title is not None:
    gv_args = gv_args + ['--title', title]
    
print()
print('Calling external process ...')
print(' '.join(gv_args))
    
_ = subprocess.call(gv_args)<jupyter_output>Discovered H2O jar path:
 C:\Users\Karti\Anaconda3\lib\site-packages\h2o\backend\bin\h2o.jar

Calling external process ...
java -cp C:\Users\Karti\Anaconda3\lib\site-packages\h2o\backend\bin\h2o.jar hex.genmodel.tools.PrintMojo --tree 0 -i C:\Users\Karti\dt_surrogate_mojo.zip -o dt_surrogate_mojo.gv --title Medical Charges Tree Surrogate<jupyter_text>Generate PNG from GraphViz representation<jupyter_code>png_file_name = model_id + '.png'
png_args = str('dot -Tpng ' + gv_file_name + ' -o ' + png_file_name)
png_args = png_args.split()

print('Calling external process ...')
print(' '.join(png_args))

_ = subprocess.call(png_args)
# for in-notebook display 
from IPython.display import Image
from IPython.display import display

display(Image((png_file_name)))
# shutdown h2o
h2o.cluster().shutdown(prompt=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11743186391073715, 'fraction_numerical': 0.07280043899762209, 'mean_word_length': 4.848128342245989, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '6832', 'license': 'MIT', 'n_tokens_mistral': 2306, 'n_tokens_neox': 1974, 'n_words': 429, 'path': 'AutoML/AutoML-Decision Tree - Insurance.ipynb', 'repo_name': 'Angelic-Interpretability/Model-Interpretability-Techniques', 'size': '5467'}