Table REF  shows results obtain for our JointCapPunc model compares with baseline model of single task training and joint model Monica et al [1]}.
For single task training model: Following common strategy when fine-tuning a pre-trained language model for token classification task [2]}. We append a linear prediction layer on the top of pre-trained language models as briefly described in Section  which is applied for the capitalization prediction task. For punctuation prediction task, we use a punctuation restore model for Vietnamese [3]} that append a CRF prediction layer on top of pre-trained language models. The single training strategy based on the XLM-R and vELECTRA
In each setting, JointCapPunc model performs better than single training approach and joint model Monica et al [1]} on both task.
Here, the highest improvements are accounted for the punctuation \(F_1\)  (ie., 78.65% \(\rightarrow \)  79.33% \(\rightarrow \)  80.10%) and the capitalization \(F_1\)  (ie., 83.55% \(\rightarrow \)  84.16% \(\rightarrow \)  86.75%).
Thus showing that our joint training model and soft capitalization features helps to better exploit the relationship between the two tasks.
Besides, vELECTRA outperforms XLM-R (punctuation mirco-\(F_1\) : 80.10% vs 78.74%, capitalization micro-\(F_1\) : 86.75% vs 85.39%).
<TABLE><TABLE>