1. We establish the effect of learning rate re-warming and re-decaying for models pre-trained using a cosine schedule, showing that re-warming and re-decaying is necessary for adaptation during continual pre-training. 2. We establish the effect of replaying previous data while keeping compute constant across two distribution shifts and many replay percentages. We find that, even when updating models on hundreds of billions of new tokens, it is possible to significantly mitigate forgetting with an appropriate amount of replay. 3. We demonstrate, across two model sizes and distribution shifts, that a simple and scalable combination of LR re-warming, LR re-decaying, and compute-equivalent replay allows continually pre-trained models to attain similar performance on average to models re-trained on the union of all data while using significantly less compute. 4. We propose infinite learning rate schedules (schedules allowing smooth transition across datasets) for the continual pre-training of LLMs as a promising way to circumvent optimization difficulties associated with learning rate re-warming.