Positional Embedding k(⋅, ⋅) The kernel construction on X = (F × T ) has distinct design in variants of <cite>Transformers</cite> <cite>(Vaswani et al., 2017</cite>; Dai et al., 2019; Huang et al., 2018b; Shaw et al., 2018; Child et al., 2019) .