Figure 2 reports the validation losses for \(\mathcal{D}_{0}\) and \(\mathcal{D}_{1}\) for all models throughout the first 50B tokens of continued pre-training on \(\mathcal{D}_{1}\). The top row reports results for the weak distribution shift, while the bottom row reports results for the strong distribution shift. Across both distribution shifts, we observe initially that models using shorter linear warmup initially forget and adapt faster than their longer warmup counterparts. This happens because they increase the LR faster which leads to faster forgetting and adaptation. In all scenarios, however, these initial differences diminish throughout training, leaving all models with relatively similar forgetting and adaptation after 50B tokens. Thus, in the settings explored, **the duration of linear warm-up phase does not appear to be impactful on forgetting or adaptation when continuing to pre-train.** With this in mind, we set a linear warmup duration of 1% of training iterations for all subsequent experiments.