As noted in prior works[1]}, [2]}, [3]}, [4]}, [5]}, many existing augmentation policies adopt random resized cropping, horizontal flipping, color jittering and so on. We generate multiple views for each image \(x\)  under random data augmentation according to multi-crop [5]}. This operation can acquire two standard resolution crops \(x_1\)  and \(x_2\)  representing the global view and sample \(N\)  low-resolution crops indicating partial view. They are encoded by two encoders, teacher network \(f_t\)  and student network \(f_s\) , parameterized by \(\theta _t\)  and \(\theta _s\)  respectively, and outputting vectors \(O_t\)  and \(O_s\) . Both encoder \(f_s\)  and \(f_t\)  consist of a Transformer backbone and a projection head [7]}, which share the same architecture with different parameters. The parameters \(\theta _t\)  of fixed encoder \(f_t\)  is updated by the moving-average of \(\theta _s\)  according to Eq (REF ).
\(\theta _t = m * \theta _t + (1 - m) * \theta _s \) 
