To further understand the parameter \(\xi \) , consider tabular RL, which can be thought of as a special case of RL under linear function approximation with \(|\mathcal {S}||\mathcal {A}|\)  feature vectors that correspond to the canonical basis vectors, i.e., \(\Phi \)  is an identity matrix. In this special case, Algorithm REF  and Theorem REF  give the finite-sample bounds of \(n\) -step off-policy tabular TD in [1]}. The actor in Algorithm REF  reduces to the NPG update [2]}. Furthermore, the function approximation bias \(\xi \)  in this case is zero, and the finite sample bounds in Theorem REF  reduce to the ones presented in [1]}.
