Figure 9: Scaling laws for GPT2 pretrained on \(\mathsf{bioS}(N)\) data with fp16 (mixed-precision) for 1000/100 exposures, **now including 1-layer transformers** comparing to Figure 1. **Conclusion: 1-layer transformers show a minor capacity ratio deficiency, especially in the 100-exposure setting.