Here, we delve into our adapter settings. For LoraHu et al. (2022), we selectively adapt only the query and value projection matrices of each layer in the model. We set the network rank to 8 and the alpha to 32. In the case of (A)3Liu et al. (2022), we rescale the key and value matrices in self-attention modules and the inner activations in feed-forward modules in each model layer via learned vectors. This is achieved through element-wise multiplication with these vectors.