We fine-tuned three versions of FlanT5 (small, base, and large) (Chung et al., 2022) model. Our fine-tuning experiments encompassed both full parameter tuning and PEFT. Our focus was particularly on the QLoRA method (Dettmers et al., 2023), which is widely recognized for its approach of keeping the pre-trained model parameters fixed while incorporating trainable rank decomposition matrices into each layer of the Transformer architecture. Despite experimenting with various numbers of training epochs, we observed that the results plateaued after a certain point. Table 5 included presents our finalized hyperparameters for fine-tuning.