The quality gap metric \(G\) reflects how much the target LLM benefits from the strong LLM for each instruction \(I\). As demonstrated in Figure 2, here are two possible cases: (1) \(|G(I)|>\theta\), where \(\theta\in\mathbb{R}\) is a certain threshold. This indicates that: Either the strong LLM has a much better response than the target LLM, we add \((I,f_{s}(I))\) to our high-quality instruction-response pool \(\mathcal{D}_{g}\) to fill the gap; Or rarely, the target LLM gives much better response than the strong LLM, we add \((I,f_{t}(I))\) to \(\mathcal{D}_{g}\) as as an implicit regularization to keep the target LLM's desirable behavior to certain instructions. (2) \(|G(I)|\leq\theta\), where the quality of responses from both LLMs is similar, so learning from \(I\) does not lead to much gain. We then send \(I\) to the next Self-Rubrics iteration for further improvement.