see [1]}, [2]} for the proof and more details. We thus employ the Burer-Monteiro factorization for all problems, except linear. For the transformer architectures, we choose the latent dimension such that the total model size is unchanged, whereas for the MLP architecture, we choose the latent dimension to have parameters on the same order as the other transformer architectures (e.g. \(b = sc/2\) ).
<TABLE>