Interpreting attention. One approach has been to experimentally observe the properties of attention networks to understand them. For example, DINO proposes a contrastive self-supervised learning method for ViTs [1]}. It is observed that learned attention maps preserve semantic regions of the images. Another work compares the alignment across layers of trained ViTs and CNNs, concluding that ViTs have more a uniform representation structure across layers of the network [2]}. Another work uses a Deep Taylor Decomposition approach to visualize portions of input image leading to a particular ViT prediction [3]}.
