@inproceedings{dykes-etal-2024-leveraging,
    title = "Leveraging High-Precision Corpus Queries for Text Classification via Large Language Models",
    author = {Dykes, Nathan  and
      Evert, Stephanie  and
      Heinrich, Philipp  and
      Humml, Merlin  and
      Schr{\"o}der, Lutz},
    editor = "Hautli-Janisz, Annette  and
      Lapesa, Gabriella  and
      Anastasiou, Lucas  and
      Gold, Valentin  and
      Liddo, Anna De  and
      Reed, Chris",
    booktitle = "Proceedings of the First Workshop on Language-driven Deliberation Technology (DELITE) @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.delite-1.7",
    pages = "52--57",
    abstract = "We use query results from manually designed corpus queries for fine-tuning an LLM to identify argumentative fragments as a text mining task. The resulting model outperforms both an LLM fine-tuned on a relatively large manually annotated gold standard of tweets as well as a rule-based approach. This proof-of-concept study demonstrates the usefulness of corpus queries to generate training data for complex text categorisation tasks, especially if the targeted category has low prevalence (so that a manually annotated gold standard contains only a small number of positive examples).",
}

We use query results from manually designed corpus queries for fine-tuning an LLM to identify argumentative fragments as a text mining task. The resulting model outperforms both an LLM fine-tuned on a relatively large manually annotated gold standard of tweets as well as a rule-based approach. This proof-of-concept study demonstrates the usefulness of corpus queries to generate training data for complex text categorisation tasks, especially if the targeted category has low prevalence (so that a manually annotated gold standard contains only a small number of positive examples).
Dykes, Nathan Evert, Stephanie Heinrich, Philipp Humml, Merlin Schr{\"o}der, Lutz
Leveraging High-Precision Corpus Queries for Text Classification via Large Language Models