Recent work has revisited these classic critiques through studies of modern neural architectures [10,<cite> 15,</cite> 3, 20, 22, 2, 6] , with a focus on the sequence-to-sequence (seq2seq) models used successfully in machine translation and other NLP tasks [32, 4, 36] . These studies show that powerful seq2seq approaches still have substantial difficulties with compositional generalization, especially when combining a new concept ("to Facebook") with previous concepts ("slowly" or "eagerly")<cite> [15,</cite> 3, 20] . New benchmarks have been proposed to encourage progress [10,<cite> 15,</cite> 2] , including the SCAN dataset for compositional learning <cite>[15]</cite> .