In practice, implementing the suggested architecture is quite straightforward: Given an auxiliary knowledge base \(\mathcal{B}_{\mathcal{Q}}\) and a pre-trained embedding model \(\mathcal{M}_{e}\), we create a dense vector representation (embedding) per document \(b\in\mathcal{B}_{\mathcal{Q}}\) and store these in a vector store. Upon receiving a new query \(q\), we use its embedding, \(\mathcal{M}_{e}(q)\), to retrieve \(q\)'s top-\(K\) closest neighbors, \(\mathbf{b}_{q}=\{b_{k}\}_{1}^{K}\), according to dot-product ranking. We then update \(q\) to be \(\tilde{q}=\mathbf{b}_{q}\|q\), where \(\|\) denotes string concatenation. Finally, we return \(\mathcal{M}(\tilde{q})\) as the model's output.