For all benchmarks in this section, we compare performance of our models with supervised learning and state-of-the-art self-supervised learning approaches on ImageNet. For each self-supervised learning approach, we chose the largest publicly available pre-trained model checkpoint and ran evaluations with them. Concretely, the models we compare with are ConvNets including SimCLRv2-RN152w3+SK  [1]} (795M params), BYOL-RN200w2  [2]} (250M params), SwAV-RN50w5 (585M params) and SwAV-RG128Gf  [3]} (693M params) and more recent Vision Transformers [4]} including MoCov3 ViT-B/16 [5]} (85M params) DINO ViT-B/16 [6]}. For SEER models, we trained several model sizes from 40M parameters to 10B parameters as described in Appendix Table REF .
<TABLE><FIGURE>