Usually, instead of retrieving just the \(\arg \max \) , the top-k scoring documents are returned.
When using cosine similarity with normalized vectors, we may substitute the similarity by the inner product \(k\, \odot \, v\) , which can be approximated efficiently by Maximum Inner Product Search (Approximate Nearest Neighbour Search) algorithms in sublinear query time [1]}, [2]}, [3]}.
Instead of documents, which are replaced by paragraphs or spans of texts, depending on the specific work.
It is also possible to store query representation from the training data, as described in subsec:kbsource. The setup is then similar as for document retrieval, with the exception that instead of documents, the retrieved values are gold outputs from the training dataset. The inner product search remains the same.
