Unlike recurrent computation (Sutskever et al., 2014 ) (i.e., RNNs) and temporal convolutional computation (Bai et al., 2018 ) (i.e., TCNs), <cite>Transformer's</cite> attention is an order-agnostic operation given the order in the inputs <cite>(Vaswani et al., 2017)</cite> .