Footnote 1: For all cosine decays in this paper, unless otherwise specified, we fit the cosine annealing phase to the token budget, set the linear warmup duration (\(T_{warmup}\)) to 1% of training iterations, and set \(\eta_{min}=0.1\cdot\eta_{max}\)