Codeswitching language identification using Subword Information Enriched Word Vectors
Codeswitching is a widely observed phenomenon among bilingual speakers. By combining subword information enriched word vectors with linear chain Conditional Random Field, we develop a supervised machine learning model that identifies languages in a English Spanish codeswitched tweets. Our computational method achieves a tweet level weighted F1 of 0.83 and a token level accuracy of 0.949 without using any external resource. The result demonstrates that named entity recognition remains a challenge in codeswitched texts and warrants further work.
CodeSwitch@EMNLP