We showed that alternating stacked encoders [1]} outperform biunidirectional stacked encoders [2]}, both in accuracy and (single-GPU) speed.
We showed that deep transition architectures, which we first applied to NMT, perform comparably to the stacked ones in terms of accuracy (Bleu, cross-entropy and long-distance syntactic agreement), and better in terms of speed and number of parameters.
