Figure 5: **Left**: Train-test similarity across validation sets. X-axis denotes the name of the validation set (refer to Section 3.4 for more information about each validation set), and y-axis denotes the cosine distance to the nearest neighbor in the training set for the 1.3B OPT 40B baseline (the green triangle denotes mean, and the yellow bar denotes median). We observe that web-snapshots validation sets are closest to points in the training set. **Right**: Analysis of the C4 validation set. (Top): Histogram of cosine distance to nearest neighbor in train. For each bin, we show the mean original perplexity (middle) and mean difference in perplexity after data selection (bottom). “Easy” (low original ppl) points close to the training set are generally the points most affected by data selection.