Afterward, for training successor models, following [28, <cite>33]</cite> , we use the first 6 layers of BERT-base to initialize the successor model since the over-parameterized nature of Transformer [38] could cause the model unable to converge while training on small datasets.