After discussing parameterization, we then analyze the effects of layer adaptation on the results by mainly focusing on Adapter-LA and Prompt-LA that both show performance gain. Figure REF  shows the effects of the rank \(r\)  in Adapter-LA and prompt length \(L\)  in Prompt-LA. As \(r\)  increases, the model performance will gradually improve. However, when \(r\)  becomes large (e.g., \(r \ge 64\) ), it will exceed our parameterization budget and thus the gain will become meaningless. As for prompt length \(L\)  in Prompt-LA, it is different from \(r\)  that it will not keep improving the results as it increases: the gain can hardly be observed after some length (e.g., \(L=8\) ), which is similar to the observation in prefix-tuning [1]}. Therefore, we use \(r=32\)  and \(L=8\)  as the default setting to report the results of Adapter-LA and Prompt-LA.
