Soft Sampling:In this case, instead of giving binary weights by leaving out all the other examples in the corpus, we assign soft weights based on the distance metric. This allows for the continual pre-training to see the samples outside the blue region in Figure 1 as well, adding some diversity to the pre-training data.