Note that, for fairness, other than manipulating the kernel choice of the non-positional features, we fix the configuration by Vaswani et al. (Vaswani et al., 2017) for NMT and the configuration by<cite> Dai et al. (Dai et al., 2019)</cite> for SP.