@inproceedings{zhang-etal-2024-chinese,
    title = "{C}hinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training",
    author = "Zhang, Longhui  and
      Long, Dingkun  and
      Zhang, Meishan  and
      Zhang, Yanzhao  and
      Xie, Pengjun  and
      Zhang, Min",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.282",
    pages = "3179--3191",
    abstract = "Chinese sequence labeling tasks are sensitive to word boundaries. Although pretrained language models (PLM) have achieved considerable success in these tasks, current PLMs rarely consider boundary information explicitly. An exception to this is BABERT, which incorporates unsupervised statistical boundary information into Chinese BERT{'}s pre-training objectives. Building upon this approach, we input supervised high-quality boundary information to enhance BABERT{'}s learning, developing a semi-supervised boundary-aware PLM. To assess PLMs{'} ability to encode boundaries, we introduce a novel {``}Boundary Information Metric{''} that is both simple and effective. This metric allows comparison of different PLMs without task-specific fine-tuning. Experimental results on Chinese sequence labeling datasets demonstrate that the improved BABERT version outperforms the vanilla version, not only in these tasks but also in broader Chinese natural language understanding tasks. Additionally, our proposed metric offers a convenient and accurate means of evaluating PLMs{'} boundary awareness.",
}

Chinese sequence labeling tasks are sensitive to word boundaries. Although pretrained language models (PLM) have achieved considerable success in these tasks, current PLMs rarely consider boundary information explicitly. An exception to this is BABERT, which incorporates unsupervised statistical boundary information into Chinese BERT{'}s pre-training objectives. Building upon this approach, we input supervised high-quality boundary information to enhance BABERT{'}s learning, developing a semi-supervised boundary-aware PLM. To assess PLMs{'} ability to encode boundaries, we introduce a novel {``}Boundary Information Metric{''} that is both simple and effective. This metric allows comparison of different PLMs without task-specific fine-tuning. Experimental results on Chinese sequence labeling datasets demonstrate that the improved BABERT version outperforms the vanilla version, not only in these tasks but also in broader Chinese natural language understanding tasks. Additionally, our proposed metric offers a convenient and accurate means of evaluating PLMs{'} boundary awareness.
Zhang, Longhui Long, Dingkun Zhang, Meishan Zhang, Yanzhao Xie, Pengjun Zhang, Min
Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training