Figure 1: A diagram illustrating the three steps of our _Reinforcement Learning from Evol-Instruct Feedback_ (_**RLEIF**_): (1) supervised fine-tuning (SFT), (2) Instruction Reward Model (IRM) training and Process-supervised Reward Model (PRM) training, and (3) Active Evol-Instruct and reinforcement learning via proximal policy optimization (PPO).