To address this drawback, [1]} propose a self-training method where the negative documents are sampled by the dense retrieval model being optimized. Since the dense retrieval model is updated in each mini-batch, the self-training follows an asynchronous iterative process  [2]}, where a set of GPUs are reserved to recalculate the dense vectors for the documents in \(\mathcal {D}\)  using the document encoder trained in the last iteration, and then update the ANN indices for negative sampling.
