Monitoring Shortcut Learning using Mutual Information
The failure of deep neural networks to generalize to out-of-distribution data
is a well-known problem and raises concerns about the deployment of trained
networks in safety-critical domains such as healthcare, finance and autonomous
vehicles. We study a particular kind of distribution shift $\unicode{x2013}$
shortcuts or spurious correlations in the training data. Shortcut learning is
often only exposed when models are evaluated on real-world data that does not
contain the same spurious correlations, posing a serious dilemma for AI
practitioners to properly assess the effectiveness of a trained model for
real-world applications. In this work, we propose to use the mutual information
(MI) between the learned representation and the input as a metric to find where
in training, the network latches onto shortcuts. Experiments demonstrate that
MI can be used as a domain-agnostic metric for monitoring shortcut learning.