Supervised: The ViT backbone is pre-trained for supervised classification using ImageNet-1k images and labels. We use the DeiT released weights [1]} for ViT-B and the ViT-L weights from [2]}, which uses an even stronger training formula than DeiT to avoid overfitting (moreover, the DeiT release does not include ViT-L). ViT-B and ViT-L were pre-trained for 300 and 200 epochs, respectively.
