roberta-base-wechsel-ukrainian

roberta-base transferred to Ukrainian using the method from the NAACL2022 paper WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.

Evaluation

Evaluation was done on lang-uk's ner-uk project, the Ukrainian portion of WikiANN and the Ukrainian IU corpus from the Universal Dependencies project. Evaluation results are the mean of 5 runs with different seeds.

Validation Results

	lang-uk NER (Micro F1)	WikiANN (Micro F1)	UD Ukrainian IU POS (Accuracy)
roberta-base-wechsel-ukrainian	88.06 (0.50)	92.96 (0.08)	98.70 (0.05)
roberta-large-wechsel-ukrainian	89.27 (0.53)	93.22 (0.15)	98.86 (0.03)
roberta-base-scratch-ukrainian*	85.49 (0.88)	91.91 (0.08)	98.49 (0.04)
roberta-large-scratch-ukrainian*	86.54 (0.70)	92.39 (0.16)	98.65 (0.09)
dbmdz/electra-base-ukrainian-cased-discriminator	87.49 (0.52)	93.20 (0.16)	98.60 (0.03)
xlm-roberta-base	86.68 (0.44)	92.41 (0.13)	98.53 (0.02)
xlm-roberta-large	86.64 (1.61)	93.01 (0.13)	98.71 (0.04)

Test Results

	lang-uk NER (Micro F1)	WikiANN (Micro F1)	UD Ukrainian IU POS (Accuracy)
roberta-base-wechsel-ukrainian	90.81 (1.51)	92.98 (0.12)	98.57 (0.03)
roberta-large-wechsel-ukrainian	91.24 (1.16)	93.22 (0.17)	98.74 (0.06)
roberta-base-scratch-ukrainian*	89.57 (1.01)	92.05 (0.09)	98.31 (0.08)
roberta-large-scratch-ukrainian*	89.96 (0.89)	92.49 (0.15)	98.52 (0.04)
dbmdz/electra-base-ukrainian-cased-discriminator	90.43 (1.29)	92.99 (0.11)	98.59 (0.06)
xlm-roberta-base	90.86 (0.81)	92.27 (0.09)	98.45 (0.07)
xlm-roberta-large	90.16 (2.98)	92.92 (0.19)	98.71 (0.04)

*trained using the same exact training setup as the wechsel-* models, but without parameter transfer from WECHSEL.

License

MIT