Ablation studies.We conduct a set of ablations of our framework to identify which factors play key roles. We evaluate two model variants trained differently than our model: _No Retriever_ trains an LM using the standard instruction-following method given instruction-output pairs, without retrieved passages; _No Critic_ trains an LM trained with input-output pairs that are always augmented with the top one retrieved document without reflection tokens. This is similar to SAIL (Luo et al., 2023), and we use our instruction-output data instead of using the Alpaca dataset (Dubois et al., 2023), as in SAIL. We also conduct ablation on our inference-time algorithm, including _No retrieval_ disables retrieval during inference; _Hard constraints_ indicates the model performance that retrieves when \(\boxed{Retrieve}\)=Yes instead of using the adaptive threshold; _Retrieve top 1_ always retrieves and uses the top one document only, similar to standard RAG approaches; _Remove_ \(\boxed{Isstr}\) indicates the model performance that removes \(\boxed{Isstr}\) score only during critique-guided beam search in Eq. 4. In this ablation experiment, we use a training instance size of 50k for a more efficient exploration of training variations. Later in this section, we conduct an analysis of the effect of training data size. We conduct the ablation studies on three datasets, PopQA, PubHealth, and ASQA. On ASQA, we evaluate models on sampled 150 instances and exclude ablations involving adaptive or no retrieval processes.