This paper presents a replicability study of the dense passage retriever (DPR) technique proposed by [1]} for end-to-end open-domain question answering (QA).
To be precise, we use the term replicability in the sense articulated by the ACM,Artifact Review and Badging characterized as “different team, different experimental setup”.
We are able to achieve comparable measurements (i.e., effectiveness on different test collections) based on an independently developed computational artifact (i.e., a different implementation).
Specifically, our experiments rely on model checkpoints shared by the original authors, but we have otherwise built an entirely different implementation (other than the evaluation scripts).
