Following recent works [1]}, we perform finetuning at input image resolution 384. Further, for iNaturalist18 full-finetuning, we initialize the model weights with SEER models and attach a linear 8142 dimensional MLP head. We finetune the full model using SGD momentum of \(0.9\)  for 48 epochs. We use learning rate of \(0.015\)  for a batchsize of 256 images, weight decay of \(1e-4\)  and use global SyncBatchNorm synchronizing the statistics across all GPU workers. We use cosine learning rate schedule decaying learning rate at every iteration to the final value of \(5e-7\) . We do not regularize BatchNorm and neither the bias in the model layers.
