Apart from the CONV-BN structure in the 4D partition of EfficientFormer, we explore Group Normalization (GN) and channel-wise Layer Normalization (LN) in the 4D partition as employed in the prior work [1]}.
By comparing V1 and V2 in Tab. REF , we can observe that the GN (V2-GN) can only slightly improve accuracy (\(0.3\%\)  top-1) but incurs latency overhead as it can not be folded at the inference stage. Similarly, applying LN (V2-LN) gets higher latency than BN while the performance improvement is negligible.
As a result, we apply the CONV-BN structure in the entire 4D partition in EfficientFormer.
