We experimented with several popular transformer models like BERT [1]}, XLNET [2]}, ALBERT [3]}, RoBERTa [4]}, SpanBERT [5]} etc. From the pre-trained transformer models we selected, we grouped the large models and base models separately in order to release two English models. A large model; en-large which is more accurate, but has a low efficiency regarding space and time. The base model; en-base is efficient, but has a comparatively low accuracy than the en-large model.
