While using knowledge bases for language modelling is not immediately intuitive, it is very prevalent in the area of open-domain question answering. Commonly in this case the retrieved knowledge (passed to a generative model) is useful for producing an answer to a question in natural language as the model is trained to condition its generation on the retrieved artefact.
Having access to a knowledge base is mandatory for the task of extractive question answering where the output is a span from the available data.
Pre-trained language models have also been shown to be able to perform question-answering in the form of being primed by the question [1]}. This demonstrates the amount of knowledge that is stored in their parameters, though these models are usually outperformed by models with explicit knowledge base access [2]}.
Models in the task of slot-filling and fact-checking also benefit from explicit memory access largely because of the same reasons.
