@inproceedings{sun-etal-2024-efficient,
    title = "Efficient and Accurate Contextual Re-Ranking for Knowledge Graph Question Answering",
    author = "Sun, Kexuan  and
      Jedema, Nicolaas Paul  and
      Sharma, Karishma  and
      Janssen, Ruben  and
      Pujara, Jay  and
      Szekely, Pedro  and
      Moschitti, Alessandro",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.496",
    pages = "5585--5595",
    abstract = "The efficacy of neural {``}retrieve and generate{''} systems is well established for question answering (QA) over unstructured text. Recent efforts seek to extend this approach to knowledge graph (KG) QA by converting structured triples to unstructured text. However, the relevance of KG triples retrieved by these systems limits their accuracy. In this paper, we improve the relevance of retrieved triples using a carefully designed re-ranker. Specifically, our pipeline (i) retrieves over documents of triples grouped by entity, (ii) re-ranks triples from these documents with context: triples in the 1-hop neighborhood of the documents{'} subject entity, and (iii) generates an answer from highly relevant re-ranked triples. To train our re-ranker, we propose a novel {``}triple-level{''} labeling strategy that infers fine-grained labels and shows that these significantly improve the relevance of retrieved information. We show that the resulting {``}retrieve, re-rank, and generate{''} pipeline significantly improves upon prior KGQA systems, achieving a new state-of-the-art on FreebaseQA by 5.56{\%} Exact Match. We perform multiple ablations that reveal the distinct benefits of our contextual re-ranker and labeling strategy and conclude with a case study that highlights opportunities for future works.",
}

The efficacy of neural {``}retrieve and generate{''} systems is well established for question answering (QA) over unstructured text. Recent efforts seek to extend this approach to knowledge graph (KG) QA by converting structured triples to unstructured text. However, the relevance of KG triples retrieved by these systems limits their accuracy. In this paper, we improve the relevance of retrieved triples using a carefully designed re-ranker. Specifically, our pipeline (i) retrieves over documents of triples grouped by entity, (ii) re-ranks triples from these documents with context: triples in the 1-hop neighborhood of the documents{'} subject entity, and (iii) generates an answer from highly relevant re-ranked triples. To train our re-ranker, we propose a novel {``}triple-level{''} labeling strategy that infers fine-grained labels and shows that these significantly improve the relevance of retrieved information. We show that the resulting {``}retrieve, re-rank, and generate{''} pipeline significantly improves upon prior KGQA systems, achieving a new state-of-the-art on FreebaseQA by 5.56{\%} Exact Match. We perform multiple ablations that reveal the distinct benefits of our contextual re-ranker and labeling strategy and conclude with a case study that highlights opportunities for future works.
Sun, Kexuan Jedema, Nicolaas Paul Sharma, Karishma Janssen, Ruben Pujara, Jay Szekely, Pedro Moschitti, Aless ro
Efficient and Accurate Contextual Re-Ranking for Knowledge Graph Question Answering