A simple method of merging multiple models is to average the weights of multiple models fine-tuned from the same base initial model. This _model soup_ approach [48] demonstrated significant improvements on relatively large image processing and image classification models. The work pointed to similarity between weighted model averaging leading to a flat local minima. Theoretical [11; 25; 26; 37] and empirical studies dating back from the 1990s [18; 19] demonstrated that flatter local optima generalize better to out-of-distribution shifts. More recent work [8; 34] examined the theoretical properties and practical implications of weight averaging applied to larger generative models.