Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post hoc Explanations
Despite the plethora of post hoc model explanation methods, the basic
properties and behavior of these methods and the conditions under which each
one is effective are not well understood. In this work, we bridge these gaps
and address a fundamental question: Which explanation method should one use in
a given situation? To this end, we adopt a function approximation perspective
and formalize the local function approximation (LFA) framework. We show that
popular explanation methods are instances of this framework, performing
function approximations of the underlying model in different neighborhoods
using different loss functions. We introduce a no free lunch theorem for
explanation methods which demonstrates that no single method can perform
optimally across all neighbourhoods and calls for choosing among methods. To
choose among methods, we set forth a guiding principle based on the function
approximation perspective, considering a method to be effective if it recovers
the underlying model when the model is a member of the explanation function
class. Then, we analyze the conditions under which popular explanation methods
are effective and provide recommendations for choosing among explanation
methods and creating new ones. Lastly, we empirically validate our theoretical
results using various real world datasets, model classes, and prediction tasks.
By providing a principled mathematical framework which unifies diverse
explanation methods, our work characterizes the behaviour of these methods and
their relation to one another, guides the choice of explanation methods, and
paves the way for the creation of new ones.