Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming
Across applications spanning supervised classification and sequential
control, deep learning has been reported to find "shortcut" solutions that fail
catastrophically under minor changes in the data distribution. In this paper,
we show empirically that DNNs can be coaxed to avoid poor shortcuts by
providing an additional "priming" feature computed from key input features,
usually a coarse output estimate. Priming relies on approximate domain
knowledge of these task-relevant key input features, which is often easy to
obtain in practical settings. For example, one might prioritize recent frames
over past frames in a video input for visual imitation learning, or salient
foreground over background pixels for image classification. On NICO image
classification, MuJoCo continuous control, and CARLA autonomous driving, our
priming strategy works significantly better than several popular
state-of-the-art approaches for feature selection and data augmentation. We
connect these empirical findings to recent theoretical results on DNN
optimization, and argue theoretically that priming distracts the optimizer away
from poor shortcuts by creating better, simpler shortcuts.