In this paper, we propose to model local and global contexts for AS2 by ensembling multiple sentences in Transformer networks [1]} and Bag-of-Word (BOW) features, respectively.
More specifically, we consider candidates as a triplet (\(s_{i-1}\) , \(s_i\) , \(s_{i+1}\) ), where \(s_i\)  is the target answer sentence and \(s_{i-1}\)  and \(s_{i+1}\)  are the preceding and the next sentence of \(s_i\) , respectively.
We integrate this triplet in Transformer architectures by using one single RoBERTa [2]} model encoding the three sentences in three embeddings. Then, we add document-level BOW representation is the classification layer.
We tested our models on three different datasets, Google NQ and SQuAD adapted for the AS2 task, as well as the well-known WikiQA, comparing with the very recent state of the art in AS2 [3]}. The results clearly show that local and global contexts can improve AS2 models.
