Reference ModelingWe begin by curating a high-quality dataset that reflects the desired data distribution. We train a reference model (RM) using standard cross-entropy loss on the curated data. The resulting RM is then used to assess the token loss within a larger pretraining corpus. We compute the reference loss (\(\mathcal{L}_{\text{ref}}\)) of a token \(x_{i}\) based on the probability that the RM assigns to this token.