Evaluating Abstractive Summarization (Part 3)
In addition to these traditional metrics, we showcase a method (G-Eval) that leverages Large Language Models (LLMs) as a novel, reference-free metric for assessing abstractive summaries. In this case, we use gpt-4 to score candidate outputs. gpt-4 has effectively learned an internal model of language quality that allows it to differentiate between fluent, coherent text and low-quality text. Harnessing this internal scoring mechanism allows auto-evaluation of new candidate outputs generated by an LLM.