The algorithm for online RL in Step 2 is given in Algorithm REF , which is an Actor-Critic method with eligibility traces combined with the linear control law [1]}, [2]}.
[t]
One-step Actor-Critic method with eligibility traces combined with linear control law

Input: \(K,\ \lbrace  \phi _{j} \rbrace _{j=1}^{N},\ \pi ,\ {\theta }_0,\ W_0,\ {x}_0\in \mathbb {R}^n\)
