To calculate carbon emissions, we multiply the total power consumption by a carbon intensity factor, measured in kg CO\({}_{2}\) emitted per KWh, based on the physical location of the data center where each model was trained. The model trained on A100-40GB GPUs was trained in Australia, so we assume a carbon intensity factor of 0.610, the national average for Australia in 2022.10 The model trained on MI250X GPUs was trained in the LUMI supercomputer, which runs on 100% renewable, carbon-neutral energy, so we assume a carbon intensity factor of 0. LUMI is powered entirely by hydroelectric power and some sources (Ubierna et al., 2022) measure the carbon intensity factor of hydroelectric power to be 0.024, which would imply total carbon emissions of 3.54 tCO\({}_{2}\)eq.11 However, we rely on the official LUMI data for our calculations, and thus we estimate total pretraining emissions of **69.78 tCO\({}_{2}\)eq.12** In Table 8 we compare our models with other previously released models based on publicly available information.