In particular, we introduce the convolutional attention, the column-wise cross-attention, and the reduction layer, which build on the Transformer model <cite>[2]</cite> to enable its application to question-answering.