We emphasize that many standard assumptions in the RL theory literature such as tabular MDPs [1]}, [2]} and Linear MDPs [3]}, [4]} are special cases of Assumption REF .
In the appendix, we consider a misspecified setting and show that even when (REF ) holds approximately, Algorithm  achieves provable regret bounds.
