(i) Absolute Positional Embedding (Vaswani et al., 2017; <cite>Dai et al., 2019</cite>; Ott et al., 2019) : For the original Transformer (Vaswani et al., 2017) , each t i is represented by a vector with each dimension being sine or cosine functions.