For contextual encoding, most of the models utilize one of the two types of encoders: (a.) Bidirectional sequential language models such as BiDAF [1]} or ELMo [2]} (b.) Deep bidirectional transformer-based models such as BERT [3]} or RoBERTa [4]}.
