Former work [1]} found that the performance will be better if dropping BN in the projection head. We argue that the degradation is not caused by BN. As shown in Table REF , normal BN downgrades the performance of baseline model, while the update rule introduced in Section REF  helps improve top-1 accuracy slightly. This may be due to the need to keep consistent structure with the global image Decoder since the image Decoder consists of Conv-BN-ReLu.
<TABLE>