Our idealized setting, free from irrelevant data, allows for more accurate scaling law computations -- we also discuss how "junk" data affects capacity later in Section 10. In contrast, it is difficult to quantify real-life knowledge; for instance, if LLaMA-70B outperforms LLaMA-7B by 30% on a benchmark, it doesn't necessarily mean a tenfold model scaling only boosts capacity by 30% (see Footnote 1).