Overall, due to the significantly different training durations (i.e., number of training tokens) across the 100-, 300-, 600-, and 1000-exposure settings, we had to adjust their batch sizes, weight decay, and learning rates accordingly. These adjustments are discussed below.