**Parameter 9** (Figure 16, Figure 17).: In the 1000-exposure setting for GPT2-MoE models with 32 experts, we slightly increase the learning rates while keeping other parameters nearly identical to Parameter 1: