We train linear classifiers by learning a multinomial logistic regression on the visual features. We initialize models with respective model weight and attach a linear classifier head initialized from scratchWe follow  [1]} which uses a BatchNorm followed by linear layer. We found that this setting leads to robust hyperparameter choice and sweeping hyperparams such as learning rate, weight decay only leads to marginal (+/-0.1) change in performance. on top. While keeping the model trunk fixed, we train the linear classifier head for 28 epochs using SGD momentum 0.9, weight decay \(0.0005\) , learning rate of \(0.01\)  for a batchsize of 256 and step learning rate schedule with gamma factor \(0.1\)  with decay at steps [8, 16, 24]. For majority of tasks above, we use the same settings and note any differences in Appendix .
