Commonly used evaluation metrics such as BLEU [1]} and ROUGE [2]} fall short at correlating with human judgments when evaluating the quality of automatically generated questions [3]}, [4]}, [5]}. [6]} carried a human evaluation based on fluency, relevance, whether the question dealt with missing information, and usefulness. In addition, [7]} proposed to assess the quality of automatically generated questions based on whether they were well-formed, concise, answerable, and answer-matching. Similarly, we conduct a human evaluation of the generated questions adapted to fact-checking.
