We validate our method from two different angles: First, we conduct a transferability study and show that our representations perform well on downstream semantic segmentation tasks.
Second, we tackle the more challenging fully unsupervised setup proposed in [1]} based on directly clustering the pixel or patch embeddings.
For that, two model characteristics are important: unsupervised foreground extraction and a semantically-structured embedding space, see Figure REF .
To our surprise, even though self-supervised ViTs excel at extracting objects, they do not learn a spatial token embedding space that is discriminative for different object categories.
On the other hand, ViTs trained under supervision achieve better semantic segmentation performance, but the attention heads perform poorly at localizing objects.
In contrast, our method outperforms self-supervised ViTs and ResNets in fully unsupervised semantic segmentation as well as in learning transferable dense representations.
