Training Objective.
Besides BYOL, ExtreMA also works with other Siamese representation learning objectives, such as InfoNCE [1]} with negatives. In Table REF , we provide the result with a MoCo-v3 [2]} implementation using a contrast temperature of \(0.2\) . Compared with the BYOL objective, the finetuning performance is improved by 0.2% and linear probing drops by 2.0%.
<TABLE><TABLE>