More recently, alternatives have been introduced for self-attention that relax the graph assumption for efficient mixing. Instead, they leverage the geometric structures using Fourier transform [1]}, [2]}. For instance, the Global Filter Networks (GFN) proposes depthwise global convolution for token mixing that enjoys an efficient implementation in the Fourier domain [1]}. GFN mainly involves three steps: \((i)\)  spatial token mixing via fast Fourier transform (FFT); \((ii)\)  frequency gating; and \((iii)\)  inverse FFT for token demixing. GFN however lacks adaptivity and expressiveness at high resolutions since the parameter count grows with the sequence size, and no channel mixing is involved in \((ii)\) .
