Some well-known publications have focused either on interpretable models (e.g., [1]}, [2]}) or on post-hoc explanations (e.g., [3]}, [4]}), but the two can be used together in the context of a holistic ML workflow, illustrated in Figure REF . Consider the seemingly useful example case of augmenting globally interpretable models with local post-hoc explanations. A practitioner could train a single decision tree, a globally interpretable model, then apply local explanations in the form of Shapley feature importance as in Figure REF .Code to replicate Figure REF : https://bit.ly/2miCPpo. This enables practitioners to see accurate numeric feature contributions for each prediction and the entire directed graph of the decision tree. Even for interpretable models, such as linear models and decision trees, Shapley values present accuracy and consistency advantages over standard feature attribution methods [5]}, [3]}, [7]}. Shapley values also enable the consistent ranking of input features for each model decision, which is likely helpful for FCRA and ECOA compliance. Another twist on the idea of combining explainable ML methods and interpretable models is described by [8]} in “Surrogate Assisted Feature Extraction for Machine Learning (SAFE ML)”. In the SAFE ML approach, features learned by more complex models are extracted and used in an explainable fashion to increase the accuracy of more interpretable models. Aren't either of these augmented processes more desirable than either an interpretable model or post-hoc explanations by themselves?
