We therefore introduce Contrastive Filtering, a method to select the instructions that can effectively enhance the target LLM \(f_{t}\). For clarity, we define the space of all natural language sequences as \(\mathcal{N}\). We have the strong LLM \(f_{s}:\mathcal{N}\rightarrow\mathcal{N}\), the target LLM \(f_{t}:\mathcal{N}\rightarrow\mathcal{N}\), and a scoring function \(S:\mathcal{N}\rightarrow\mathbb{R}\) to evaluate response quality. In practice, \(S\) is obtained by reusing the strong LLM \(f_{s}\) with a prompt template (Figure 11, Appendix A.9) adapted from the Vicuna pairwise evaluation template (Taori et al., 2023; Chiang et al., 2023). To mitigate potential position bias, we average the scores obtained by exchanging the positions of two responses (Chiang et al., 2023). We observe using \(f_{s}\) for scoring works quite well in practice, so we prioritize this option for simplicity. Given an input instruction \(I\in\mathcal{N}\), we obtain responses from both LLMs as \(f_{s}(I)\) and \(f_{t}(I)\), respectively. We then define the _quality gap_\(G:\mathcal{N}\rightarrow\mathbb{R}\) between these responses to estimate the _effectiveness_ of the instruction: \(G(I)=S(f_{s}(I))-S(f_{t}(I))\).