The most prevalent LMs nowadays are the GPT families (Radford et al., 2019) (Brown et al., 2020) (Achiam et al., 2023) and Bert families (Liu et al., 2019) (Devlin et al., 2018)(Sanh et al., 2019), which have been demonstrated to excel in a multitude of natural language processing tasks. Among these, the AutoEncoder language model is particularly adept at natural language understanding tasks,while the AutoRegressive language model is more suited to natural language generation tasks. While increasing parameters (Touvron et al., 2023b) and model tuning (Han et al., 2023) can enhance the performance of LLMs, the phenomenon of "hallucination" (Ji et al., 2023) persists. Furthermore, the limitations of LMs in effectively handling knowledge-intensive work (Feng et al., 2023) and their inability to promptly update their knowledge (Mousavi et al., 2024) are consistently apparent. Consequently, numerous researchers (Lewis et al., 2020) (Izacard and Grave, 2020b) (Khandelwal et al., 2019) have employed the technique of retrieval to obtain external knowledge, which can assist the language model in attaining enhanced performance in a multitude of tasks.