We conduct experiments on our ViCapPunc dataset to study the effectiveness of joint training model. Here, we employ XLM-R [1]} is pre-trained on a 2.5TB multilingual dataset that contains 137 GB Vietnamese text and vELECTRA [2]}, a monolingual variant of ELECTRA for Vietnamese, is pre-trained on 10GB of Vietnamese text.
