Footnote 25: Selecting the best LoRA option is justified as our aim is to determine the maximum extractable knowledge bits, and thus, any LoRA option demonstrating high test-set accuracy fulfills our objective.