As we pointed out in Sec. REF , one of the key components of contrastive methods like SimCLR is the large numbers of the negative examples, i.e. SimCLR used a large mini-batch size and MoCo used a large memory bank to maintain. While the proposed methods in SimCLR and MoCo achieved good performance, the improvement of the performance diminishes with the increase in the amount of negative examples. Why does this happen? \(\dagger \)  Mixing of contrastive hard negatives (MoCHi) [1]} MoCHi is the main artifact in this subsection. investigated into this phenomenon and showed that the negative examples are not utilized fully. Thus, MoCHi provided a latent space mixing strategy to increase the contribution of the negative examples and thus improve the performance of contrastive learning method.
