@inproceedings{di-buono-etal-2024-evaluating,
    title = "Evaluating Large Language Models for Linguistic Linked Data Generation",
    author = "di Buono, Maria Pia  and
      Spahiu, Blerina  and
      Barbu Mititelu, Verginica",
    editor = "S{\'e}rasset, Gilles  and
      Oliveira, Hugo Gon{\c{c}}alo  and
      Oleskeviciene, Giedre Valunaite",
    booktitle = "Proceedings of the Workshop on Deep Learning and Linked Data (DLnLD) @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.dlnld-1.6",
    pages = "66--75",
    abstract = "Large language models (LLMs) have revolutionized human-machine interaction with their ability to converse and perform various language tasks. This study investigates the potential of LLMs for knowledge formalization using well-defined vocabularies, specifically focusing on OntoLex-Lemon. As a preliminary exploration, we test four languages (English, Italian, Albanian, Romanian) and analyze the formalization quality of nine words with varying characteristics applying a multidimensional evaluation approach. While manual validation provided initial insights, it highlights the need for developing scalable evaluation methods for future large-scale experiments. This research aims to initiate a discussion on the potential and challenges of utilizing LLMs for knowledge formalization within the Semantic Web framework.",
}

Large language models (LLMs) have revolutionized human-machine interaction with their ability to converse and perform various language tasks. This study investigates the potential of LLMs for knowledge formalization using well-defined vocabularies, specifically focusing on OntoLex-Lemon. As a preliminary exploration, we test four languages (English, Italian, Albanian, Romanian) and analyze the formalization quality of nine words with varying characteristics applying a multidimensional evaluation approach. While manual validation provided initial insights, it highlights the need for developing scalable evaluation methods for future large-scale experiments. This research aims to initiate a discussion on the potential and challenges of utilizing LLMs for knowledge formalization within the Semantic Web framework.
di Buono, Maria Pia Spahiu, Blerina Barbu Mititelu, Verginica
Evaluating Large Language Models for Linguistic Linked Data Generation