In this paper, we first investigate convergence differences between the published Transformer <cite>(Vaswani et al., 2017)</cite> and the official implementation of the Transformer , and compare the differences of computation orders between them.