Simple weight interpolation works well for merging image generation models that operate in a real valued latent and output space, but to our surprise, they also still work for merging language models under the right conditions, but have performance issues [49]. Recently, new methods have been proposed to address merging language models specifically. One method, Task Arithmetic [21], involves building _task vectors_ by subtracting pre-trained from fine-tuned model weights, enabling manipulation through arithmetic operations to steer the merged model's behavior.