Several works have studied this problem and the predominant finding is that LMs struggle to answer questions about documents they have been trained on Wang et al. (2021); Zhu and Li (2023). It is important to note, however, that these experiments were mainly conducted using relatively small LMs such as BART, T5, or GPT-2 Wang et al. (2021); Jang et al. (2022); Hu et al. (2023), using randomly initialized transformers Zhu and Li (2023), or without instruction-tuning Ovadia et al. (2023). This makes us wonder _what are the actual limits of modern LLMs to absorb knowledge from new documents and answer questions about them using the standard continued pre-training followed by instruction-tuning recipe_. In this section, we run extensive experiments using Llama-2 7B and 70B on Wiki2023-film to test their limits.