* For presenting _negative_ results (Figure 5(a) and Figure 5(c)), we select the best learning rate from three options, identical to GPT2\({}_{0}\) in Parameter 4(c). * For presenting _positive_ results (Figure 5(b) and Figure 5(d)), we use a single set of learning rates, identical to Parameter 2 but with fp16 replaced by bf16 for a stronger comparison.