@inproceedings{marmol-romero-etal-2024-environmental,
    title = "Environmental Impact Measurement in the {M}ental{R}isk{ES} Evaluation Campaign",
    author = "M{\'a}rmol Romero, Alba M.  and
      Moreno-Mu{\~n}oz, Adri{\'a}n  and
      Plaza-del-Arco, Flor Miriam  and
      Molina Gonz{\'a}lez, M. Dolores  and
      Montejo-R{\'a}ez, Arturo",
    editor = "Gaspari, Federico  and
      Moorkens, Joss  and
      Aldabe, Itziar  and
      Farwell, Aritz  and
      Altuna, Begona  and
      Piperidis, Stelios  and
      Rehm, Georg  and
      Rigau, German",
    booktitle = "Proceedings of the Second International Workshop Towards Digital Language Equality (TDLE): Focusing on Sustainability @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.tdle-1.6",
    pages = "61--72",
    abstract = "With the rise of Large Language Models (LLMs), the NLP community is increasingly aware of the environmental consequences of model development due to the energy consumed for training and running these models. This study investigates the energy consumption and environmental impact of systems participating in the MentalRiskES shared task, at the Iberian Language Evaluation Forum (IberLEF) in the year 2023, which focuses on early risk identification of mental disorders in Spanish comments. Participants were asked to submit, for each prediction, a set of efficiency metrics, being carbon dioxide emissions among them. We conduct an empirical analysis of the data submitted considering model architecture, task complexity, and dataset characteristics, covering a spectrum from traditional Machine Learning (ML) models to advanced LLMs. Our findings contribute to understanding the ecological footprint of NLP systems and advocate for prioritizing environmental impact assessment in shared tasks to foster sustainability across diverse model types and approaches, being evaluation campaigns an adequate framework for this kind of analysis.",
}

With the rise of Large Language Models (LLMs), the NLP community is increasingly aware of the environmental consequences of model development due to the energy consumed for training and running these models. This study investigates the energy consumption and environmental impact of systems participating in the MentalRiskES shared task, at the Iberian Language Evaluation Forum (IberLEF) in the year 2023, which focuses on early risk identification of mental disorders in Spanish comments. Participants were asked to submit, for each prediction, a set of efficiency metrics, being carbon dioxide emissions among them. We conduct an empirical analysis of the data submitted considering model architecture, task complexity, and dataset characteristics, covering a spectrum from traditional Machine Learning (ML) models to advanced LLMs. Our findings contribute to understanding the ecological footprint of NLP systems and advocate for prioritizing environmental impact assessment in shared tasks to foster sustainability across diverse model types and approaches, being evaluation campaigns an adequate framework for this kind of analysis.
M{\'a}rmol Romero, Alba M. Moreno-Mu{\~n}oz, Adri{\'a}n Plaza-del-Arco, Flor Miriam Molina Gonz{\'a}lez, M. Dolores Montejo-R{\'a}ez, Arturo
Environmental Impact Measurement in the MentalRiskES Evaluation Campaign