Recent work has revisited these classic critiques through studies of modern neural architectures [10,<cite> 15,</cite> 3, 20, 22, 2, 6] , with a focus on the sequence-to-sequence (seq2seq) models used successfully in machine translation and other NLP tasks [32, 4, 36] .