For retrieval, we pre-compute the token encodings of all the documents in the corpus, and use ScaNN [1]} to index and perform max inner-product search (MIPS). We retrieve 4,000 nearest neighbors for each query token,Unlike ColBERT, AligneR does not use pad token embeddings for retrieval. Hence, retrieving 4,000 neighbors per query token results in a similar number of retrieved candidates to ColBERT. and return the top-1,000 after the refinement stage. We evaluate AligneR on the BEIR benchmark [2]} and compare with state-of-the-art retrieval models shown in tab:comparisons.
Note that AligneR does not rely on contrastive model pre-training [3]}, [4]}, model-based hard negative mining [5]}, or distillation [5]}. We intentionally decide this simple recipe and focus on studying the impact of pairwise alignment and unary salience.
