RoBERTa-Large-Chunk simply chunks passages into pieces, which is a common practice for end-to-end MRC models [1]}, [2]}. It obtains a question-level accuracy of 41.9%, lagging behind the methods with additional evidence extraction step by at least 5.7%.
We think it is because
the evidence sentences of a statement may appear in different chunks.
Without evidence selection, models have to split the long article, which results in
incomplete evidence for training and harm the performance.
Even such simple methods as BM25 outperform the chunking method thanks to the retrieved evidence that is more relevant to the statement.
Moreover, by introducing evidence selection, we can have a small-sized but focused supportive candidates to be feed the MRC model, greatly reducing required computational resources.
These results illustrate the necessity of selecting evidence pieces from the long articles before performing verification.
<TABLE>