**Rationale Evaluation and Refinement.** In contexts beyond rationale distillation, there have been growing efforts to unravel which aspects of the generated reasoning steps contribute to the downstream task performance. Wang et al. (2022a) report that rationales' logicality and relevance to the query are key factors in successful CoT reasoning. Few studies have measured the validity of reasoning steps from the lens of more diverse aspects like informativeness, coherence, and repetition, etc (Golovneva et al., 2022; Prasad et al., 2023). While RL has gained popularity as an approach for addressing misaligned behaviors in LMs, the field of rationale correction has seen limited research.