Note that we directly used the results of BERT on SST-2, MNLI, QNLI and MRPC which are reported by<cite> Devlin et al. (2019)</cite> and reimplemented the BERT model fine-tuned on the rest of the tasks by ourselves.