For perplexity, we notice in Figure A4 that the claims in Section 4 generally hold across validation sets: for web snapshots validation sets such C4, CC-dedup, and CommonCrawl, we see performanceworsens with data selection compared to baseline training, and that D4 generally has the slowest rate of performance degradation. We note that, across all non web-snapshot validation sets, there is no clear winner among data selection methods. We emphasize however that _we observe consistent improvement over baseline training on most validation sets_ we use -- for example in Figure A4 we observe that, when selecting tokens from a 1.25x source dataset, all data selection methods improve over baseline across all validation sets except C4 and CC-dedup (however, as we explain in Section 4.4, this decrease in performance on C4 and CC-dedup is expected).