paragraph4
.5em plus1ex minus.2ex-.5emDiscussion. The COCO dataset is a challenging setting for transfer learning. Due to the large training set (\(\scriptstyle \sim \) 118k images with \(\scriptstyle \sim \) 0.9M annotated objects), it is possible to achieve strong results when training from random initialization. We find that existing methods, like supervised IN1k or unsupervised MoCo v3 pre-training, actually underperform the AP of the random initialization baseline (though they yield faster convergence). Prior works reporting unsupervised transfer learning improvements on COCO (, [1]}) tend to show modest gains over supervised pre-training (, \(\scriptstyle \sim \) 1 AP\(^\text{box}\) ) and do not include a strong random initialization baseline as we do here (because strong training formulae based on large-scale jitter had not yet been developed). Moreover, they use weaker models and report results that are overall much lower (, \(\scriptstyle \sim \) 40 AP\(^\text{box}\) ) making it unclear how well the findings translate to state-of-the-art practices.
