Training high-quality visual representations that work well on a wide range of downstream tasks has been a core interest in the computer vision community.
Recent advances in self-supervised learning [1]}, [2]}, [3]}, [4]}, [5]} have shown that high quality visual features can be trained without labels.
They surpass the performance of supervised learning on many computer vision tasks including object detection, image classification and low-shot learning.
