Our study primarily addressed a template-based QA task, suggesting future research could tackle more complex QA challenges, such as multi-hop QA Ho et al. (2020) or Conversational QA Christmann et al. (2022). Furthermore, we noted that our use of Zephyr for synthetic data generation has limitations in following Chain of Thought (CoT), highlighting the potential benefits of advanced data generation techniques to enhance data quality and reduce fine-tuning cost.