For the 100-exposure scaling law, we additionally compare three configurations: \((topk,cap\_factor)=(1,2),(2,1),(2,2)\), finding minimal differences among them as shown in Figure 17. Remember from Section 7 that differences in model architecture usually become apparent in the insufficient training regime; this is why we opt for 100-exposure instead of 1000-exposure. Notably, \((topk,cap\_factor)=(2,2)\) performs best (among the three) for deep models, such as GPT2-16-4 with 32 experts.