While large pretrained models offer significantly increased performance, they come with their own constraints, as the number of parameters in the <cite>classic BERT-base model</cite> exceeds 100 million.