
where \(\mathbf {x}_1\)  and \(\mathbf {x}_2\)  are the contextual representations of \(X_1\)  and \(X_2\) , generated by the two encoders (dual-encoding), respectively, and sim(.) is a similarity function which can be implemented using cosine similarity or a model parameterized by \(\theta \) .
Compared to the pair prediction formulation, dual-encoding is more often used for document retrieval where runtime efficiency is important. Dual-encoding allows documents and queries to be encoded separately so that we can pre-encode all documents and the IR system only needs to encode an input query on the fly for retrieval. Dual-encoding is widely used for dense retrieval [1]}, [2]}, [3]}, [4]}, which will be discussed in later sections.
This formulation is also referred to as the representation-based model, dual-encoder, siamese-encoder, weak interaction model, deep structured similarity model etc.
