In Figure REF , we plot exact match scores as a function of varying \(k\)  retrieval results for NQ (left) and TriviaQA (right).
That is, we show how end-to-end QA effectiveness changes as the reader is provided more contexts from the retriever to consider.
There are two factors here at play: On the one hand, top-\(k\)  accuracy increases monotonically, i.e., as \(k\)  increases, so does the likelihood that the answer appears in the contexts fed to the reader.
On the other hand, the reader is asked to consider more contexts, and thus needs to discriminate the correct answer from a larger pool of candidate contexts, some of which might be low quality and thus serve as “distractors” from the correct answer.
How do these factors balance out?
Similar analyses in previous work with BM25 retrieval have shown that end-to-end QA effectiveness increases with increasing \(k\)  [1]}, [2]}; that is, the reader does not appear to be “confused” by the non-relevant material.
Indeed, in our BM25 results we also observe the same trend.
