Even though these implicit methods are efficient methods of injecting or modifying knowledge from the implicit parameters of the LMs, they are all limited to injecting _specific knowledge_ such as the case of (Wang et al., 2021) or modifying _past knowledge_ such as the case of (Zhu et al., 2020; De Cao et al., 2021). No work, to the best of our knowledge, has specifically addressed the _catastrophic forgetting_ of world knowledge gained from the initial pretraining when continued pretraining on new text corpus for the gain of _new_ world knowledge.