Figure 9: **In our core aggregate, deduplication brings a small improvement to the Pythia suite (Biderman et al., 2023).** Zero-shot performance on our core aggregate, gap between Pythia trained on the deduplicated and vanilla Pile. Individual curves for per-task results and 1-\(\sigma\) standard deviation across all tasks in the aggregate in transparent.