ESTIMATING THE CARBON FOOTPRINT OF BLOOM,
A176B P ARAMETER LANGUAGE MODEL
Alexandra Sasha Luccioni
Hugging Face
sasha.luccioni@h
Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model
Progress in machine learning (ML) comes with a cost to the environment, given
that training ML models requires significant computational resources, energy
and materials. In the present article, we aim to quantify the carbon footprint
of BLOOM, a 176-billion parameter language model, across its life cycle. We
estimate that BLOOM's final training emitted approximately 24.7 tonnes
of~\carboneq~if we consider only the dynamic power consumption, and 50.5 tonnes
if we account for all processes ranging from equipment manufacturing to
energy-based operational consumption. We also study the energy requirements and
carbon emissions of its deployment for inference via an API endpoint receiving
user queries in real-time. We conclude with a discussion regarding the
difficulty of precisely estimating the carbon footprint of ML models and future
research directions that can contribute towards improving carbon emissions
reporting.
Alexandra Sasha Luccioni Sylvain Viguier Anne-Laure Ligozat