Related work includes that of [1]}, who have performed an exploration of NMT architectures in parallel to our work.
Their experiments, which are largely orthogonal to ours, focus on embedding size, RNN cell type (GRU vs. LSTM), network depth (defined according to the architecture of [2]}), attention mechanism and beam size.
[3]} recently proposed a NMT architecture based on convolutions over fixed-sized windows rather than RNNs, and they reported results for different model depths and attention mechanism configurations.
A similar feedforward architecture which uses multiple pervasive attention mechanisms rather than convolutions was proposed by [4]}, who also report results for different model depths.
