However, even with residual connections and layer normalization, deep Transformers are still hard to train: the original Transformer <cite>(Vaswani et al., 2017)</cite> only contains 6 encoder/decoder layers.