We use Mask-RCNN [1]} for all detection and instance segmentation experiments.
We pretrain the backbone on ImageNet, mostly reusing the same hyperparameters as in Section REF .
Backbones are pretrained for 350 epochs using an image size of 512, which was chosen to be closer to the 1024 image size used in detection setting.
The models were regularized with RandAug at a magnitude of 15 and stochastic depth with probability \(0.1\) , and use Squeeze-Excitation with a reduction factor of \(\frac{1}{8}\) .
The detection code and hyperparameters directly used the open-source TPU detection and segmentation framework
https://github.com/tensorflow/tpu/tree/master/models/official/detection.
During the detection / instance segmentation phase, the backbone is initialized with the pretrained weights, while the other parameters are initialized from scratch.
The model is trained for 67500 steps with \(0.1\) x learning rate decays at 60000 and 65000 steps, uses a learning rate of \(0.1\)  in SGD with \(0.9\)  momentum, a warmup of 500 steps with a fixed learning rate of \(\frac{2}{300}\) , a batch size of 64 spread across 32 TPUv3 cores, \(1024\times 1024\)  image size, an L2 weight decay of \(4e^{-5}\) , and multi-scale jitter with magnitudes between \([\frac{4}{5},\frac{5}{4}]\) .
