In contrast to self-attention or MLP-like attention mechanisms, there is also a family of Fourier-based alternatives to self-attention which have recently shown promise in vision. We present Fourier Neural Operator (FNO) [1]}, [2]}, which works as follows: \(i)\)  2D DFT is applied first over the spatial tokens; \(ii)\)  each token is multiplied by its own weight matrix; and \(iii)\)  inverse DFT returns the Fourier tokens back to the original (spatial) domain.
