Despite the significant progress, existing works in DML often focus on designing high-level learning objective functions, while ignoring the low-level but essential data transformation. In fact, data transformation has played a crucial role in recently advanced methods in self-supervised representation learning. In contrastive learning, SimCLR [1]} experimentally demonstrated that the composition of multiple data transformations is of central importance in learning effective representations. Moreover, in pretext learning, it yields a series of pretext tasks to predict what the specific transformation is applied to an image, e.g., rotation estimation [2]} and patch orderings prediction [3]}.
