**K-Adapter**(Wang et al., 2021) Similarly with T5-LoRA, we freeze the encoder for the encoder-decoder LM and the entire model for GPT-2. We implement \(k=2,3\) for both T5 and GPT-2 to see the effect of increasing # of parameters. Unlike in the original paper, we set the configuration of the adapter identical to a single transformer layer from the original LM, ridding the need of an up-projection and down-projection layer.