v1 and v2 stand for the computation order of the proposed Transformer <cite>(Vaswani et al., 2017)</cite> and that of the official implementation respectively. "¬" means fail to converge, "None" means not reported in original works, "*" indicates our implementation of their approach.