included in the original training corpus is supported by the low QA performance in Tab. 1 (9.5%/17.2% for 7B/70B).4 To accelerate the training process, we only use the first section of each article, which offers a thorough summary and contains many factual statements. The number of collected documents and an example document about "Oppenheimer" can be found in Fig. 2 and Fig. 3. We refer to this as the Wiki2023 dataset.