[1]} incorporates the training strategies from MoCoV2 [2]} and BYOL [3]}. Instead of using vanilla ViT as the backbone, MoBY replaces it with Swin-Transformer [4]}. Additionally, the updating strategy is similar to DINO. On top of that, to avoid a large batch size, MoBY follows MoCoV2 to create memory queue for reusing the past features.
