\(\mathbf {BERT}\)  [1]} is a pre-trained model proposed by Google in 2017, which adopts a masked language model to the bidirectional transformer to finish pre-training tasks. Then the last few layers of model parameters need to be fine-tuned to achieve satisfactory results.
