Another method to enhance a model's knowledge base is through the use of in-context learning (ICL) (Chen et al., 2021; Radford et al., 2019; Min et al., 2021; Lampinen et al., 2022). The main idea behind ICL is to improve the performance of pre-trained LLMs on new tasks by modifying the input query to the model without directly changing the weights of the model. One form of ICL is retrieval augmented generation (RAG) (Lewis et al., 2020; Neelakantan et al., 2022). RAG uses information retrieval techniques to enable LLMs to obtain relevant information from a knowledge source and incorporate it into generated text.