When processing language, humans selectively attend longer to the most relevant elements of a sentence [1]}. This ability to seamlessly evaluate relative importance is a key factor in human language understanding. It remains an open question how relative importance is encoded in computational language models. Recent analyses conclude that the cognitively motivated “attention” mechanism in neural models is not a good indicator for relative importance [2]}. Alternative methods based on salience [3]}, vector normalization [4]}, or subset erasure [5]} are being developed to increase the post-hoc interpretability of model predictions but the cognitive plausibility of the underlying representations remains unclear.
