DPR uses two BERT models to compute the document and question embeddings at the [CLS] token (768-dimensional). They are then fine-tuned so that the inner product (or \(L^2\)  distance or cosine similarity) of these two vectors is a good measure for document relevancy.
The retrieved passages are reranked by a combination of the original similarity and the BM25 model [1]}.
For every retrieved passage, the probability of a span (up to fixed maximum length) being selected is computed as the product of two tokens (representation from another BERT model) being the start and end ones, respectively.
Reranking of the answers is done implicitly by choosing the span with the highest probability across all spans.
The maximum similarity search is approximated to make it computationally feasible using FAISS [2]}.
\(& \texttt {Encoder:} & k = \textsc {Bert}_\text{query}(q)\\& \texttt {Knowledge base:} & \mathcal {B} = \lbrace  \textsc {Bert}_\text{doc}(d) |d \in \mathcal {KB} \rbrace  \\& \texttt {Retriever:} & C = \arg \max ^n_{d\in B} \text{sim}(q, d) \\& & C^{\prime } = \text{Reranker}(C) \\& & \text{score given by: } \textsc {BM25}(q,d) + \lambda \cdot \text{sim}(q,d) \\& \texttt {Model:} & P_{\text{start},i}(s) = \text{softmax}(C^{\prime }_i w_\text{start})_s \\& & P_{\text{end},i}(t) = \text{softmax}(C^{\prime }_i w_\text{end})_t \\& & P_{\text{selection}}(i) = \text{softmax}(C^{\prime } w_\text{selection})_i \\\) 
