@inproceedings{gromann-etal-2024-multilexbats,
    title = "{M}ulti{L}ex{BATS}: Multilingual Dataset of Lexical Semantic Relations",
    author = "Gromann, Dagmar  and
      Goncalo Oliveira, Hugo  and
      Pitarch, Lucia  and
      Apostol, Elena-Simona  and
      Bernad, Jordi  and
      Byty{\c{c}}i, Eliot  and
      Cantone, Chiara  and
      Carvalho, Sara  and
      Frontini, Francesca  and
      Garabik, Radovan  and
      Gracia, Jorge  and
      Granata, Letizia  and
      Khan, Fahad  and
      Knez, Timotej  and
      Labropoulou, Penny  and
      Liebeskind, Chaya  and
      Di Buono, Maria Pia  and
      Ostro{\v{s}}ki Ani{\'c}, Ana  and
      Rackevi{\v{c}}ien{\.e}, Sigita  and
      Rodrigues, Ricardo  and
      S{\'e}rasset, Gilles  and
      Selmistraitis, Linas  and
      Sidib{\'e}, Mahammadou  and
      Silvano, Purifica{\c{c}}{\~a}o  and
      Spahiu, Blerina  and
      Sogutlu, Enriketa  and
      Stankovi{\'c}, Ranka  and
      Truic{\u{a}}, Ciprian-Octavian  and
      Valunaite Oleskeviciene, Giedre  and
      Zitnik, Slavko  and
      Zdravkova, Katerina",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1029",
    pages = "11783--11793",
    abstract = "Understanding the relation between the meanings of words is an important part of comprehending natural language. Prior work has either focused on analysing lexical semantic relations in word embeddings or probing pretrained language models (PLMs), with some exceptions. Given the rarity of highly multilingual benchmarks, it is unclear to what extent PLMs capture relational knowledge and are able to transfer it across languages. To start addressing this question, we propose MultiLexBATS, a multilingual parallel dataset of lexical semantic relations adapted from BATS in 15 languages including low-resource languages, such as Bambara, Lithuanian, and Albanian. As experiment on cross-lingual transfer of relational knowledge, we test the PLMs{'} ability to (1) capture analogies across languages, and (2) predict translation targets. We find considerable differences across relation types and languages with a clear preference for hypernymy and antonymy as well as romance languages.",
}

Understanding the relation between the meanings of words is an important part of comprehending natural language. Prior work has either focused on analysing lexical semantic relations in word embeddings or probing pretrained language models (PLMs), with some exceptions. Given the rarity of highly multilingual benchmarks, it is unclear to what extent PLMs capture relational knowledge and are able to transfer it across languages. To start addressing this question, we propose MultiLexBATS, a multilingual parallel dataset of lexical semantic relations adapted from BATS in 15 languages including low-resource languages, such as Bambara, Lithuanian, and Albanian. As experiment on cross-lingual transfer of relational knowledge, we test the PLMs{'} ability to (1) capture analogies across languages, and (2) predict translation targets. We find considerable differences across relation types and languages with a clear preference for hypernymy and antonymy as well as romance languages.
Gromann, Dagmar Goncalo Oliveira, Hugo Pitarch, Lucia Apostol, Elena-Simona Bernad, Jordi Byty{\c{c}}i, Eliot Cantone, Chiara Carvalho, Sara Frontini, Francesca Garabik, Radovan Gracia, Jorge Granata, Letizia Khan, Fahad Knez, Timotej Labropoulou, Penny Liebeskind, Chaya Di Buono, Maria Pia Ostro{\v{s}}ki Ani{\'c}, Ana Rackevi{\v{c}}ien{\.e}, Sigita Rodrigues, Ricardo S{\'e}rasset, Gilles Selmistraitis, Linas Sidib{\'e}, Mahammadou Silvano, Purifica{\c{c}}{\~a}o Spahiu, Blerina Sogutlu, Enriketa Stankovi{\'c}, Ranka Truic{\u{a}}, Ciprian-Octavian Valunaite Oleskeviciene, Giedre Zitnik, Slavko Zdravkova, Katerina
MultiLexBATS: Multilingual Dataset of Lexical Semantic Relations