**Data Augmentation (DA).** Data availability is crucial for fine-tuning in specialized domains. DA addresses data scarcity problem by generating task- and domain-relevant samples from existing unlabeled texts. A common DA approach for the QA task is generating question-answer pairs through a four-step _Pipeline_, consisting of: passage selection, answer extraction, question generation, and consistency filtering Alberti et al. (2019); Lewis et al. (2019, 2021); Ushio et al. (2023). Ushio et al. (2023) conducted an empirical study comparing three question answer generation approaches: Pipeline, Multitask, and End-to-End (E2E) and showed the E2E approach outperforms others in downstream tasks. Recently, the utilization of LLMs to generate data is shown effective in information retrieval, QA, and dialogue creation tasks Soudani et al. (2023); Askari et al. (2023).