To the best of our knowledge, this is the first result to demonstrate the benefits of repeating data for LLM pre-training, over randomly sampling new tokens via a principled data selection technique. We argue that the optimal way of using large-scale web data to pre-train LLMs could be: strategically choose a significantly smaller but better-distributed subset of the data and epoch over it multiple times.