paragraph4
.5em plus1ex minus.2ex-.5emConvergence. In Figure REF  we show how pre-training impacts fine-tuning convergence. Given the tuned hyperparameters for each initialization method, we train models for \(2\times \)  and \(4\times \)  longer (and also \(0.5\times \)  for random initialization). Generally, we find that all pre-trained initializations significantly accelerate convergence compared to random initialization, as observed in [1]}. Most methods show signs of overfitting when the training schedule is made sufficiently long, typically by 100 epochs for pre-trained initializations and 400 epochs for random initialization. Based on this data, pre-training tends to accelerate training on COCO by roughly \(4\times \)  compared to random initialization.
