The reliance on training data is taken to its extremes by one of the models in [1]} in their study of overlap (in terms of paraphrases) between test and train datasets for question answering.
In their approach, they simply use nearest neighbours to retrieve the closest paraphrase to the query (using vector space model) to answer the question.
The encoding is done either by the pretrained DPR retriever [2]} or by TF-IDF.
As a consequence model can easily answer a question granted that the paraphrase is present in the training set.
\(& \texttt {Encoder:} & k = \textsc {DPR}_\text{query}(q)\\& \texttt {Knowledge base:} & \mathcal {B} = \lbrace  (\textsc {DPR}_\text{query}(d) |d \in \mathcal {KB} \rbrace  \\& \texttt {Retriever:} & o = \xi = \arg \max _{d\in B}\, \langle q, d\rangle \) 
