Transformer, proposed by Vaswani et al. [1]}, are ubiquitous in NLP tasks [2]}, [3]}, [4]}. Fig. REF  illustrates the architecture of Transformer. A single transformer block can be separated into Encoder and Decoder, which both can be further decomposed into 1) self-attention, 2) Position-wise Feed-Forward Networks, and 3) positional encoding.
