Evaluation protocols.
For all experiments, we discard the projection head used during training.
Instead we directly evaluate the ViT's spatial tokens.
We use two main techniques for evaluation: linear classifier and overclustering.
For linear classifier (LC), we fine-tune a 1x1 convolutional layer on top of the frozen spatial token or the pre-GAP layer4 features, following [1]}.
For overclustering, we run K-Means on all spatial tokens of a given dataset.
We then group cluster to ground-truth classes by greedily matching by pixel-wise precision and run Hungarian matching [2]} on the merged cluster maps to make our evaluation metric permutation-invariant following [3]}.
We always report overclustering results averaged over five different seeds.
Overclustering is of special interest as it works directly in the learned embedding space and therefore requires less supervision than training a linear classifier.
For completeness we also report results fine-tuning a deeper fully-convolutional net (FCN) following [4]}.
Generally, we follow the fine-tuning procedures of prior works [5]}, [1]}, [4]}.
We report results in mean Intersection over Union (mIoU) unless specified otherwise.
