In the presence of the success of pre-trained language models, especially BERT <cite>[2]</cite> , it is natural to ask how to best utilize the pre-trained language models to achieve new state-of-the-art results.