
Self-Attention [1]} uses 16 attention heads and a hidden size of 768.

Long-Short transformer [2]} (LS) uses a window-size of 4 and dynamic projection rank of 8 with a hidden size of 768.

Global Filter Network [3]} (GFN) uses a hidden size of 768.

Adaptive Fourier Neural Operator (AFNO) uses 1 block, a hidden dimension of 750, and a sparsity threshold of \(0.1\) , and a 1D convolution layer as the bias.

