Compared to deep transition encoder alone, we generally see improvements in cross-entropy, but not in Bleu.
We evaluate architectures similar to [1]} (alternating encoder + stacked GRU decoder) and [2]} (biunidirectional encoder + stacked rGRU decoder), though they are not straight replications since we used GRU cells rather than LSTMs and the implementation details are different.
We find that the former architecture performs better in terms of Bleu scores, model size and training speed.
