The quality of the trained model depends on the quality and quantity of the training data. One common approach is to collect weak relevance labels in large quantities from search click logs. For example, [1]} treat all the documents that have been clicked more than 5 times by users for query \(Q\)  as its relevant (positive) documents \(D^+\) , and randomly sample a set of documents from the document collection \(\mathcal {D}\)  as irrelevant (negative) documents \(D^-\) .
However, such randomly sampled \(D^-\) 's are often so irrelevant to \(Q\)  (i.e., resulting in \(\Delta \gg 0\)  in Equations REF  and REF ) that they do not provide useful information for model training [2]}.
[3]} provides a theoretical analysis on the impact of negative sampling for dense retrieval model training using the variance reduction and negative sampling framework [4]}, [5]}. The analysis shows that randomly sampling negatives from \(\mathcal {D}\)  is unlikely an effective importance sampling strategy and often yields inefficient learning or slow convergence.
