Note that, for fairness, other than manipulating the kernel choice of the non-positional features, we fix the configuration by <cite>Vaswani et al</cite>. <cite>(Vaswani et al., 2017)</cite> for NMT and the configuration by Dai et al. (Dai et al., 2019) for SP.