Negative Sampling. Hard negative mining has been shown to be crucial for contrastive learning [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}. [3]} use the time difference between clips to approximate clip similarity (i.e., clips that are further apart are deemed more different). However, such an assumption may not hold for real-world videos, e.g., periodic actions such as push-ups. Unlike this line of approaches, we directly use the feature embeddings learned by our model. Several apparoaches adapted a similar idea [4]}, [5]}, [6]}, [7]}. Different from prior work, we bring the stochasticity to the sampling procedure by using the content similarity as the sampling probability; this helps reduce potential errors especially during the early stage of training.
