Besides, we also adopt Mask2Former [1]} to evaluate the pre-trained models on ADE-20K semantic segmentation. We use an AdamW [2]} optimizer with a base learning rate of 1e-4 for supervised model and a base learning rate of 3e-4 for MIM model, a weight decay of 0.05 and a patch size of 16 for both supervised and MIM models. The learning rate of backbone is multiplied by a factor of 0.1. All models are trained for 160K iterations with an input resolution of 512 \(\times \)  512, a scale ratio range from 0.5 to 2, a window size of 8, and a drop path rate of 0.3. In inference, the input resolution will be set to 2048 \(\times \)  512. mIoU is reported for comparison for both UPerNet and Mask2Former.
