We use the original ViT formulation as proposed by [1]} and we follow the pre-training hyperparameters of [2]}. All baselines reported use the same backbone implementation and trained in similar settings. For SplitMask, by default, we use random block masking [2]} of 50% masking ratio to obtain a mask and its complement to extract the two subsets. The maximum and minimum number of patches per block is 75 and 16 respectively. We use the standard random cropping and horizontal flipping as data augmentations. We use 2 transformer layers for the decoder with embedding dimension matching that of the encoder.
