(i) Absolute Positional Embedding <cite>(Vaswani et al., 2017</cite>; Dai et al., 2019; Ott et al., 2019) : For the original <cite>Transformer</cite> <cite>(Vaswani et al., 2017)</cite> , each t i is represented by a vector with each dimension being sine or cosine functions.