In this section, we show the training performance of ViT-B models pretrained on an image reconstruction loss taken from SimMIM [1]} and DeiT-S [2]} models trained from scratch on ImageNet [3]}. We use the magnitude patch sorting function and cyclic patch sampling schedule for all PSS settings. Since SimMIM only requires 100 epochs of finetuning (due to pretraining), the ViT-B settings finish substantially faster than the DeiT ones. This faster training time is why we chose to use the ViT-B model to evaluate the impact of PSF and PSS combinations in our main evaluation. All DeiT models were trained for 300 epochs to match the DeiT experimental setup. The 224 or 384 in each model name denote the image size input to the model during training (e.g., DeiT-S-224 uses 224\(\times \) 224 and DeiT-S-384 uses 384\(\times \) 384).
