Fourier-Based Mixers apply the Fourier transform to spatially mix tokens. FNet [1]} resembles the MLP-mixer with token mixer simply being pre-fixed DFT. No filtering is done to adapt the data distribution. Global filter networks (GFNs) [2]} however learn Fourier filters to perform depthwise global convolution, where no channel mixing is involved. Also, GFN filters lack adaptivity that could negatively impact generalization. In contrast, our proposed AFNO performs global convolution with dynamic filtering and channel mixing that leads to better expressivity and generalization.
