In table (1) we show the WER of the proposed transformer encoder-decoder model with convolutional context using the canonical configuration in the first row. Replacing the 1-D convolutional context in the decoder with sinusoidal positional embedding, as proposed in the baseline machine translation transformers <cite>[11]</cite> and adopted in [13, 15] , shows inferior WER performance. By combining sinusoidal and convolutional position embedding (rows 1+2), we don't observe any gains. This supports our intuition that the relative convolutional positional information provides sufficient signal for the transformer layers to recreate more global word order.