Consistency regularization: Consistency regularization is a method of regularization that seeks to make the output of a model consistent across small perturbations in either the input [1]} or the model parameters [2]}. Recently, BYOL [3]} applied a variant of Mean Teachers [2]} for self-supervised learning. Our method uses a variation of this idea through similarity based loss rather than regression loss defined on data points individually.
