Figure REF  illustrates the
pre-training and fine-tuning framework.
This is an instance of transfer learning, where the knowledge learned by the PLM for a language modeling task (e.g., masked language modeling) is transferred to perform a different downstream task.
In this section we review the masked language modeling task used for BERT pre-training [1]}, and then discuss several common setups to fine-tune PLMs to downstream tasks.
