_Remark 1.4_.: **Our framework offers a principled playground to compare models.** This contrasts with traditional comparisons based on loss/perplexity, which can produce debatable conclusions.6 Controlled data also reveal more significant differences between models.7 Footnote 6: A model might achieve better perplexity by performing _much better_ on simpler data but slightly poorer on complex data, or by excelling in reasoning tasks but not in knowledge storage. Our results offer a more nuanced view: GatedMLP doesn’t affect frequently encountered knowledge (with 1000 exposures) but does impact moderately rare knowledge (with 100 exposures). * Section 8: How quantization affects model capacity. We applied GPTQ[10] to quantize models from the base scaling laws to int8 or int4. Surprisingly,
* Result 8: Quantizing to int8 does not compromise model capacity (even for models on the boundary of 2bit/param); however, quantizing to int4 reduces capacity to 0.7bit/param.