PG Methods. The first PG algorithm with function approximation was proposed in [1]}, where the asymptotic convergence was established. A refined asymptotic analysis of PG methods has been further proposed in [2]}, [3]}, [4]}. Natural policy gradient (NPG), which is a PG method with preconditioning, was proposed in [5]}. Recently, there has been a line of work to establish finite-sample convergence bound of NPG. In particular, sublinear convergence of NPG was established in [6]}, [7]}, [8]}, [9]}, [10]}, and geometric convergence of NPG was established in [11]}, [12]}, [13]}, [14]}, [15]}.
