We use the ImageNet-1k for MIM pre-training and augment the data using the random resize cropping (scale range \([0.67, 1]\)  and aspect ratio range \([3/4, 4/3]\) ), and random flipping. The optimizer is the AdamW[1]} optimizer with a weight decay of \(5e\) -2 and a base learning rate of \(4e\) -4. We use warm-up for 10 epochs, drop the learning rate to \(4e\) -5 at 260th epoch, and train for 300 epochs in total. The batch size is 2048. We use the DropPath of \(0.1\)  for RepLKNet-31B and gradient clipping.
