Cross-Encoder (CE)  and Dual-Encoder (DE)  models are two fundamental approaches for
predicting
query-document relevance in information retrieval.
To predict relevance,
CE models usejointquery-document embeddings,
while
DE models maintainfactorizedquery-document embeddings;
usually, the former has higher quality while
the latter has lower latency.
Recently,late-interactionmodels have been proposed to realize more favorable latency-quality trade-offs, by using a DE structure followed by a lightweight scorer based on query and document token embeddings.
However,
these lightweight scorers are often hand-crafted, and there is no understanding of their approximation power;
further, such scorers require access to individual document token embeddings, which imposes an increased latency and storage burden over DE models.
In this paper, we propose novellearnablelate-interaction models (LITE)  that resolve these issues.
Theoretically,
we prove that LITE is a universal approximator of continuous scoring functions,
even for relatively small embedding dimension.
Empirically, LITE outperforms previous late-interaction models such as ColBERT on both in-domain and zero-shot re-ranking tasks.
For instance, experiments on MS MARCO passage re-ranking show that
LITE not only yields a model with better generalization, but also lowers latency and requiresstorage compared to ColBERT.