@inproceedings{wang-etal-2024-towards-realistic,
    title = "Towards More Realistic {C}hinese Spell Checking with New Benchmark and Specialized Expert Model",
    author = "Wang, Yue  and
      Zheng, Zilong  and
      Li, Juntao  and
      Liu, Zhihui  and
      Chang, Jinxiong  and
      Zhang, Qishen  and
      Liu, Zhongyi  and
      Zhang, Guannan  and
      Zhang, Min",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1440",
    pages = "16570--16580",
    abstract = "Large Language Models (LLMs) hold considerable promise for artificial general intelligence, given their intrinsic abilities to accomplish a wide range of open-domain tasks either independently or in tandem with specialized expert models. However, despite these capabilities, the performance of LLMs has yet to be comprehensively evaluated in realistic scenarios. To this end, in this work, we introduce a novel task, the \textbf{R}ealistic \textbf{C}hinese \textbf{S}pell \textbf{C}hecking (\textbf{RCSC}), to evaluate the effectiveness of existing methods comprehensively. In contrast to existing works that solely address Chinese character misspellings or pinyin conversions, our task aims to convert the realistic Chinese text into the corresponding correct text. The realistic Chinese text may potentially contain both Chinese misspellings and pinyin conversions. We first present the \textbf{R}ealistic \textbf{C}hinese \textbf{S}pell \textbf{C}hecking \textbf{B}enchmark (\textbf{RCSCB}), which consists of two subsets and contains a total of 581,657 samples. Then, we benchmark the performance of various baselines and find that all the existing methods, including instruction-based LLMs, achieve unsatisfactory results on RCSCB. To further improve the performance on RCSCB, we propose \textbf{P}inyin-\textbf{E}nhanced \textbf{S}pell \textbf{C}hecker (\textbf{PESC}), which is specifically designed to address pinyin-related misspellings. Experimental results demonstrate that PESC can achieve state-of-the-art performance on RCSCB. Despite the progress made, the current state-of-the-art performance is still far from satisfactory. We expect further progress on this crucial and challenging task.",
}

Large Language Models (LLMs) hold considerable promise for artificial general intelligence, given their intrinsic abilities to accomplish a wide range of open-domain tasks either independently or in tandem with specialized expert models. However, despite these capabilities, the performance of LLMs has yet to be comprehensively evaluated in realistic scenarios. To this end, in this work, we introduce a novel task, the \textbf{R}ealistic \textbf{C}hinese \textbf{S}pell \textbf{C}hecking (\textbf{RCSC}), to evaluate the effectiveness of existing methods comprehensively. In contrast to existing works that solely address Chinese character misspellings or pinyin conversions, our task aims to convert the realistic Chinese text into the corresponding correct text. The realistic Chinese text may potentially contain both Chinese misspellings and pinyin conversions. We first present the \textbf{R}ealistic \textbf{C}hinese \textbf{S}pell \textbf{C}hecking \textbf{B}enchmark (\textbf{RCSCB}), which consists of two subsets and contains a total of 581,657 samples. Then, we benchmark the performance of various baselines and find that all the existing methods, including instruction-based LLMs, achieve unsatisfactory results on RCSCB. To further improve the performance on RCSCB, we propose \textbf{P}inyin-\textbf{E}nhanced \textbf{S}pell \textbf{C}hecker (\textbf{PESC}), which is specifically designed to address pinyin-related misspellings. Experimental results demonstrate that PESC can achieve state-of-the-art performance on RCSCB. Despite the progress made, the current state-of-the-art performance is still far from satisfactory. We expect further progress on this crucial and challenging task.
Wang, Yue Zheng, Zilong Li, Juntao Liu, Zhihui Chang, Jinxiong Zhang, Qishen Liu, Zhongyi Zhang, Guannan Zhang, Min
Towards More Realistic Chinese Spell Checking with New Benchmark and Specialized Expert Model