In the 1000-exposure setting, we observe that the model architecture choices have a _negligible impact_ on the scaling laws. The results for LLaMA, Mistral, GPT2\({}_{0}\), and GPT2\({}_{1/4}\) architectures are presented in Figure 3, with their parameter choices discussed below.