Setup. The CoG benchmark consists of five 1k-category datasets split from ImageNet-21K, which has an increasing semantic gap with ImageNet-1K, from \(L_1\)  to \(L_5\) .
On the CoG dataset, we search for the best hyper-parameters based on the top-1 accuracy of the \(L_1\)  validation set and then apply the best setting to CoG \(L_2\)  to \(L_5\)  to report the top-1 accuracy.
On the K12 dataset, we adopt standard splits of train/val/test sets as in [1]}. We use the training set to fine-tune the models, use the validation set to search for the best hyper-parameters, and then train the models on the merged training and validation sets using the best setting. Following [1]}, we report mean-per-class accuracy for Aircraft, Pets, Caltech-101, Oxford 102 Flowers and top-1 accuracy for other datasets.
The iNat18 dataset includes 437,513 training images and 24,426 validation images, with more than 8,000 categories. We fine-tune the pre-trained models using the training set and report the top-1 accuracy on the validation set.
For all datasets, we choose learning rate, weight decay, layer decay, and DropPath [3]} on the valid set respectively for the MIM pre-trained model and the supervised pre-trained model. We use the AdamW optimizer [4]} and cosine learning rate schedule. We train the model for 100 epochs with 20 warm-up epochs. The input image size is \(224 \times 224\) . Other detailed setups of these datasets are in the Appendix.
