Remark It was argued in [1]} that sample complexity is not well-defined when the convergence error does not go to zero. Therefore, one should not use sample complexity when we do not have global convergence due to the function approximation bias. However, we present Corollary REF  in terms of “sample complexity” in the same sense as used in prior literature to enable a fair comparison.
See Appendix  for a more detailed discussion.

