**K-Adapter**(Wang et al., 2021) is another parameter-expansion method that freezes the original parameters of the LM while adding \(k\) number of new layers, namely _adapters_, that are updated during continued pretraining. Wang et al. (2021) have shown successful injection of _factual_ and _linguistic_ knowledge for encoder-only models, BERT (Devlin et al., 2019) & RoBERTa (Liu et al., 2019), while we also apply it to an encoder-decoder model, T5, and decoder-only model, GPT-2.