We are now witnessing the growth of unsupervised learning with Vision Transformer (ViT) models [1]}, and while the high-level transfer learning methodology remains the same, the low-level details and baselines for some important downstream tasks have not been established. Notably, object detection, which has played a central role in the study of transfer learning over the last decade (, [2]}, [3]}, [4]}, [5]}), was not explored in the pioneering work on ViT training [1]}, [7]}, [8]}—supervised or unsupervised—due to the challenges (described shortly) of integrating ViTs into common detection models, like Mask R-CNN [9]}.
