The parameterized knowledge in the model is fixed at the end of the pre-training, which leads to its shortcomings in terms of knowledge capacity and knowledge updating. While introducing external knowledge can alleviate this to some extent, it remains an imperfect solution. To fundamentally tackle this issue, continual learning Lange et al. (2022); Wang et al. (2023) stands as a promising avenue for future research endeavors.