[1]} is designed to use convolutional layers in FFN to extract the local features in every transformer blocks. The authors also try to apply different activation functions (ReLU6, h-swish) and architectures (SE-Block, ECA module) in FFN layers to improve the performance.
