Where \(w_{1:T}\) is sampled from a fixed SFT model \(p_{\text{sft}}\) in the first stage, and the new data is added to the original data for the second SFT stage. While \(p\) in Eq. (3) is updated online. Singh et al. (2023) also proposes to do this two-stage method for multiple iterations.