Table 2 shows the BERT base models, including the original BERT-base model in <cite>(Devlin et al., 2018)</cite> and our implementation, and the bidirectional LSTM model accuracy over SQuAD 1.1 development dataset.