From the perspective of model structure, some recent works [1]}, [2]}, [3]} have pointed out that, an important factor that affects performance and stability of ViTs is the patchify stem implemented by a stride-\(p\)  \(p \times p\)  convolution (\(p = 16\)  by default) on the input image. To address this problem, MocoV3 [1]} froze the patch projection to train ViTs, while Xiao  [2]} and Wang  [3]} proposed a convolution stem stacked by several convolution, Batch Normalization (BN) [7]}, and ReLU [8]} layers to increase optimization stability and improve performance. Inspired by the success of integrating Instance Normalization (IN) and BN to learn domain-invariant representation in the ReID task [9]}, [10]}, [11]}, we refer to IBN-Net [10]} and improve the convolution stem to the IBN-based convolution stem (ICS). ICS inherits the stability of the convolution stem and also introduces IBN to learn features with appearance invariance (viewpoint, pose and illumination invariance, etc.). ICS has similar computational complexity to convolution stem, but improves peak performance of ViTs significantly in person ReID.
