
Self-Attention [1]} uses 12 attention heads and a hidden size of 324.

Long-Short transformer [2]} (LS) uses a window-size of 4 and dynamic projection rank of 8 with a hidden size of 312.

Global Filter Network [3]} (GFN) uses a hidden size of 245. The hidden size is smaller due to the need to make all the models have the same parameter count.

Adaptive Fourier Neural Operator (AFNO) uses 16 blocks, a hidden dimension of 384, sparsity threshold of \(0.1\) , and a 1D convolution layer as the bias.

