In the past year, the field of Natural Language Processing (NLP) has seen the rise of pretrained language models such as as ELMo (Peters et al., 2018) , ULMFiT (Howard and Ruder, 2018) and <cite>BERT</cite> (<cite>Devlin et al., 2019</cite>) . The benefit of <cite>these models</cite> has also been demonstrated in specialized NLP domains.