Attributions Beyond Neural Networks: The Linear Program Case
Linear Programs (LPs) have been one of the building blocks in machine
learning and have championed recent strides in differentiable optimizers for
learning systems. While there exist solvers for even high-dimensional LPs,
understanding said high-dimensional solutions poses an orthogonal and
unresolved problem. We introduce an approach where we consider neural encodings
for LPs that justify the application of attribution methods from explainable
artificial intelligence (XAI) designed for neural learning systems. The several
encoding functions we propose take into account aspects such as feasibility of
the decision space, the cost attached to each input, or the distance to special
points of interest. We investigate the mathematical consequences of several XAI
methods on said neural LP encodings. We empirically show that the attribution
methods Saliency and LIME reveal indistinguishable results up to perturbation
levels, and we propose the property of Directedness as the main discriminative
criterion between Saliency and LIME on one hand, and a perturbation-based
Feature Permutation approach on the other hand. Directedness indicates whether
an attribution method gives feature attributions with respect to an increase of
that feature. We further notice the baseline selection problem beyond the
classical computer vision setting for Integrated Gradients.