First, we find that using more contexts as inputs do not always improve the results. Utilizing top-5 contexts as input yields better results compared to using either top-3 or top-10 contexts. Intuitively, more contexts have higher probability to contain correct answers (better recall scores). As a result, using top-5 contexts achieves better results than using top-3. However, as the number of contexts further increases, the model may suffer from "lost in the middle" phenomenon (Liu et al., 2023) and the difficulty of extracting answers from the provided context could also increase, which leads to inferior results by using top-10 contexts.