Language models utilize contextualized word representations to boost the performance of various NLP tasks [1]}, [2]}, [3]}, [4]}. In recent years, there has been a rise in the trend of integrating external knowledge into language models [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]} based on the transformer [12]}. For instance, representations of entities in the input and related concepts are combined with contextual representations to provide additional information, leading to significant improvement in many tasks [13]}, [14]}, [15]}, [16]}.
