However, it remains largely unclear how ratio clipping relates to KL regularization,
and whether/how a trust region constraint is implicitly enforced via ratio clipping.
Existing studies, including the original PPO paper [1]},
interpret this ratio clipping as a way to bound the ratios [2]}, [3]}.
Many other studies, however, suggest contradictory conclusions:
the ratios could become unbounded via ratio clipping [4]}, [5]}, [6]},
and, even if the ratios are bounded, a KL-based trust region constraint could still be violated [4]}.
Furthermore, several studies point out that RC-PPO may benefit from code-level optimizations,
rather than ratio clipping itself [5]}, [9]}.
