The Transformer <cite>(Vaswani et al., 2017)</cite> , which has outperformed previous RNN/CNN based translation models (Bahdanau et al., 2014; Gehring et al., 2017) , is based on multi-layer self-attention networks and can be trained very efficiently.