We additionally evaluate the translation ability of our models through the FLORES-101 (Goyal et al., 2021) benchmark. The ability to translate between languages is a good test of the model's ability to relate and understand semantic relationships between languages.