Existing multi-vector retrieval models such as ColBERT [1]} computes query-document similarity by selecting the highest scoring document token for each query token and aggregating the scores.
This sum-of-max method has two major limitations.
First, restricting the selection to a single document token can be highly sub-optimal for some retrieval tasks.
As we will show in our experiments, the retrieval performance can be improved by more than 16 points nDCG@10 by relaxing this constraint.
Second, the method also leads to a large search index and expensive computation.
Specifically, the retrieval and storage cost scales linearly with the query and document length, making multi-vector retrieval models an inferior choice for efficiency-demanding applications.
We directly tackle these challenges to build faster and more accurate models.
