Traditionally, question-answering and other NLP systems would use a vector space model for the values (documents) and queries. Such representations usually describe the algorithm for computing the vector representations for both the document and the query. These include standard statistical methods such as TF-IDF [1]} or BM25 [2]}. Their disadvantage is that the resulting vectors have the dimension the same as the number of words in the vocabulary. A contrast to this is dense vector representation. LSA [3]} is one of the oldest of such methods, but it also includes learned embedding methods based on word2vec [4]}, doc2vec [5]}, docT5query [6]}, CLSM [7]} or BERT [8]}. The latter allows for the use of the gradient signal of the latent variable (documents) to fine-tune the index, as done asynchronously by [9]}.
