Transformer(Vaswani et al., 2017) is an "encoder-decoder" architecture, which consists of encoders and decoders superimposed on multi-head self-attention modules. Among them, the input sequence is divided into two parts, the source sequence and the destination sequence. The former is input to the encoder and the latter is input to the decoder, and both sequences need to embed representation and add position information. The Transformer architecture enables parallel computation and the processing of entire text sequences simultaneously, resulting in a significant increase in model training and inference speed.