_Remark B.1_ (bf16 on gated MLP).: As discussed in Section B.2, the training of LLaMA and Mistral architectures is less stable due to the use of GatedMLP, leading to the necessity of switching to (mixed-precision) bf16 training when required.