VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives
Many past works aim to improve visual reasoning in models by supervising
feature importance (estimated by model explanation techniques) with human
annotations such as highlights of important image regions. However, recent work
has shown that performance gains from feature importance (FI) supervision for
Visual Question Answering (VQA) tasks persist even with random supervision,
suggesting that these methods do not meaningfully align model FI with human FI.
In this paper, we show that model FI supervision can meaningfully improve VQA
model accuracy as well as performance on several Right-for-the-Right-Reason
(RRR) metrics by optimizing for four key model objectives: (1) accurate
predictions given limited but sufficient information (Sufficiency); (2)
max-entropy predictions given no important information (Uncertainty); (3)
invariance of predictions to changes in unimportant features (Invariance); and
(4) alignment between model FI explanations and human FI explanations
(Plausibility). Our best performing method, Visual Feature Importance
Supervision (VisFIS), outperforms strong baselines on benchmark VQA datasets in
terms of both in-distribution and out-of-distribution accuracy. While past work
suggests that the mechanism for improved accuracy is through improved
explanation plausibility, we show that this relationship depends crucially on
explanation faithfulness (whether explanations truly represent the model's
internal reasoning). Predictions are more accurate when explanations are
plausible and faithful, and not when they are plausible but not faithful.
Lastly, we show that, surprisingly, RRR metrics are not predictive of
out-of-distribution model accuracy when controlling for a model's
in-distribution accuracy, which calls into question the value of these metrics
for evaluating model reasoning. All supporting code is available at
https://github.com/zfying/visfis