Previously Seen KnowledgeOne important distinction to make is between knowledge that the model has been exposed to before during pre-training as opposed to entirely new facts. Considering the size of modern LLM training sets, they cover a vast amount of information available through web-sourced text. As a result, even in niche domains, the goal of knowledge injection is not necessarily to teach the model entirely new facts but rather to "refresh" its memory by inducing a bias toward a particular domain.