Code-switching is a phenomenon in which multilingual speakers switch between languages within a single utterance. Previous work on the performance of multilingual language models on code-switching tasks has shown mixed results. Some studies have suggested that pretrained models fine-tuned for specific code-switching scenarios can achieve state-of-the-art performance for certain language pairs such as English-Spanish and English-Hindi Khanuja et al. (2020), while others have found that using meta-embeddings can yield better results with fewer parameters Winata et al. (2019); Winata et al. (2019, 2021). In another line of research, code-switching-based methods have been presented to improve the capability of multilingual language models Jiang et al. (2020); Tan and Joty (2021); Krishnan et al. (2021).