sketch for thm: main for mixed case  
We first connect the regret with the sum of advantages, \(\sum _{t=1}^T (V^\star (s_t) - Q^\star (s_t,a_t))\) . This is standard based on the performance difference lemma [1]}.
