
The first term on the RHS of Eq. (REF ) represents the convergence rate of the actor while the second term is a combination of the error in the critic estimate and the function approximation bias. This already improves the result in [1]}, where they have \(\mathcal {O}(1/\sqrt{T})\)  convergence rate of the actor in the function approximation setting while we have \(\mathcal {O}(1/T)\) . Note that the \(\mathcal {O}(1/T)\)  convergence rate matches with the convergence rate of the actor in the tabular setting [1]}, [3]}, [4]}.
