@inproceedings{alam-etal-2024-towards,
    title = "Towards Realistic Few-Shot Relation Extraction: A New Meta Dataset and Evaluation",
    author = "Alam, Fahmida  and
      Islam, Md Asiful  and
      Vacareanu, Robert  and
      Surdeanu, Mihai",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1442",
    pages = "16592--16606",
    abstract = "We introduce a meta dataset for few-shot relation extraction, which includes two datasets derived from existing supervised relation extraction datasets {--} NYT29 (Takanobu et al., 2019; Nayak and Ng, 2020) and WIKI- DATA (Sorokin and Gurevych, 2017) {--} as well as a few-shot form of the TACRED dataset (Sabo et al., 2021). Importantly, all these few-shot datasets were generated under realistic assumptions such as: the test relations are different from any relations a model might have seen before, limited training data, and a preponderance of candidate relation mentions that do not correspond to any of the relations of interest. Using this large resource, we conduct a comprehensive evaluation of six recent few-shot relation extraction methods, and observe that no method comes out as a clear winner. Further, the overall performance on this task is low, indicating substantial need for future research. We release all versions of the data, i.e., both supervised and few-shot, for future research.",
}

We introduce a meta dataset for few-shot relation extraction, which includes two datasets derived from existing supervised relation extraction datasets {--} NYT29 (Takanobu et al., 2019; Nayak and Ng, 2020) and WIKI- DATA (Sorokin and Gurevych, 2017) {--} as well as a few-shot form of the TACRED dataset (Sabo et al., 2021). Importantly, all these few-shot datasets were generated under realistic assumptions such as: the test relations are different from any relations a model might have seen before, limited training data, and a preponderance of candidate relation mentions that do not correspond to any of the relations of interest. Using this large resource, we conduct a comprehensive evaluation of six recent few-shot relation extraction methods, and observe that no method comes out as a clear winner. Further, the overall performance on this task is low, indicating substantial need for future research. We release all versions of the data, i.e., both supervised and few-shot, for future research.
Alam, Fahmida Islam, Md Asiful Vacareanu, Robert Surdeanu, Mihai
Towards Realistic Few-Shot Relation Extraction: A New Meta Dataset and Evaluation