**500K length book summarization (BookSum).** We further scaled our approach by continuously pre-training a 8B LLM model with 8K input length for 30K steps. We then fine-tuned on a book summarization task, BookSum (Kryscinski et al., 2021) where the goal is to generate a summary of an entire book text.