Another characteristic of explainable models are the assumptions that they make about knowledge of the original model structure. Some approaches assume knowledge of the original model composition (white box assumption) while other approaches only assume that access to inputs and output are available from the full model (black box assumption)~([1]}). Approaches that make black-box assumptions are also referred to as model-agnostic, in the sense that they can be used without detailed knowledge of the original models structure. For example, Local Interpretable Model-agnostic Explanations~() (LIME) is a recent example of a model agnostic approach that creates locally-optimized explanations by training a surrogate model.
