Full results are summarised in Table 10. One may notice that our reported forward task accuracies are slightly higher than those reported in Allen-Zhu & Li (2023a). This improvement is attributed to our use of a larger batch size, smaller weight decay, and the best result among three runs.