@inproceedings{beks-van-raaij-etal-2024-clearer,
    title = "Clearer Governmental Communication: Text Simplification with {C}hat{GPT} Evaluated by Quantitative and Qualitative Research",
    author = "Beks van Raaij, Nadine  and
      Kolkman, Daan  and
      Podoynitsyna, Ksenia",
    editor = "Nunzio, Giorgio Maria Di  and
      Vezzani, Federica  and
      Ermakova, Liana  and
      Azarbonyad, Hosein  and
      Kamps, Jaap",
    booktitle = "Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.determit-1.15",
    pages = "152--178",
    abstract = "This research investigates the application of ChatGPT for the simplification of Dutch government letters, aiming to enhance their comprehensibility without compromising legal accuracy. We use a three-stage mixed method evaluation procedure to compare the performance of a naive approach, RoBERTA, and ChatGPT. We select the six most complicated letters from a corpus of 200 letters and use the three approaches to simplify them. First, we compare their scores on four evaluation metrics (ROUGE, BLEU, BLEURT, and LiNT), then we assess the simplifications with a legal and linguistic expert. Finally we investigate the performance of ChatGPT in a randomized controlled trial with 72 participants. Our findings reveal that ChatGPT significantly improves the readability of government letters, demonstrating over a 20{\%} increase in comprehensibility scores and a 19{\%} increase in correct question answering among participants. We also demonstrate the importance of a robust evaluation procedure.",
}

This research investigates the application of ChatGPT for the simplification of Dutch government letters, aiming to enhance their comprehensibility without compromising legal accuracy. We use a three-stage mixed method evaluation procedure to compare the performance of a naive approach, RoBERTA, and ChatGPT. We select the six most complicated letters from a corpus of 200 letters and use the three approaches to simplify them. First, we compare their scores on four evaluation metrics (ROUGE, BLEU, BLEURT, and LiNT), then we assess the simplifications with a legal and linguistic expert. Finally we investigate the performance of ChatGPT in a randomized controlled trial with 72 participants. Our findings reveal that ChatGPT significantly improves the readability of government letters, demonstrating over a 20{\%} increase in comprehensibility scores and a 19{\%} increase in correct question answering among participants. We also demonstrate the importance of a robust evaluation procedure.
Beks van Raaij, Nadine Kolkman, Daan Podoynitsyna, Ksenia
Clearer Governmental Communication: Text Simplification with ChatGPT Evaluated by Quantitative and Qualitative Research