Machine common sense has long been acknowledged as a critical component for natural language understanding. The challenge for the model has turned to abstractive knowledge comprehension. A task that is closely related to SemEval 2020 Task 4 is CommonsenseQA [1]}, in which the commonsense knowledge is required to make the correct prediction. In CommonsenseQA task, large-scale pre-trained models have brought significant performance gains. These gains are obtained by developing training strategies and enlarging training data [2]}, or improving parameter efficiency [3]}.
