Contrastive Filtering complements Self-Rubrics to select effective instruction-response pairs by calibrating the target LLM's instruction-following capability with the strong LLM's. Analogous to Contrastive Decoding (Li et al., 2022) at response-level, Contrastive Filtering can also be regarded as LLM-feedback (Madaan et al., 2023) with the interaction of two LLMs. While we adopt the strong LLM as scoring function to measure the quality gap, our framework can be compatible with and potentially benefit from the advances in more reliable and comprehensive scoring and feedback systems (Lee et al., 2023), and we leave it as promising future work.