The NMT models are generally constructed with up to 6 encoder and decoder blocks in both state-of-the-art research work and champion systems of machine translation competition. For example, the LSTM-based models are usually stacked for 4 (Stahlberg et al., 2018 ) or 6 (Chen et al., 2018 blocks, and the state-of-the-art Transformer models are equipped with a 6-block encoder and decoder<cite> (Vaswani et al., 2017</cite>; JunczysDowmunt, 2018; Edunov et al., 2018) .