Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images
A key concern in integrating machine learning models in medicine is the
ability to interpret their reasoning. Popular explainability methods have
demonstrated satisfactory results in natural image recognition, yet in medical
image analysis, many of these approaches provide partial and noisy
explanations. Recently, attention mechanisms have shown compelling results both
in their predictive performance and in their interpretable qualities. A
fundamental trait of attention is that it leverages salient parts of the input
which contribute to the model's prediction. To this end, our work focuses on
the explanatory value of attention weight distributions. We propose a
multi-layer attention mechanism that enforces consistent interpretations
between attended convolutional layers using convex optimization. We apply
duality to decompose the consistency constraints between the layers by
reparameterizing their attention probability distributions. We further suggest
learning the dual witness by optimizing with respect to our objective; thus,
our implementation uses standard back-propagation, hence it is highly
efficient. While preserving predictive performance, our proposed method
leverages weakly annotated medical imaging data and provides complete and
faithful explanations to the model's prediction.