To better discuss the accuracy improvement brought by this approach, we test the proposed PIM on four mainstream backbones. As shown in Table REF , the four mainstream backbones, including ResNet-50[1]}, EfficientNet[2]}, Vit[3]}, and Swin-T[4]} can be improved after adding our proposed PIM. The accuracy rate of Swin-T before joining PIM has exceeded state-of-the-art approaches, which shows that when the capacity of the backbone is good enough, the performance in downstream tasks will be quite good. So it can be predicted that these fine-grained recognition tasks will also become easier to complete after pre-training on large datasets through self-supervised methods. In this experiments, the pre-training dataset for EfficientNet-7B is ImageNet-1K, while the others are ImageNet-21K or -22K.
<TABLE><TABLE><TABLE><FIGURE>