In existing literature, to achieve stability in the presence of the deadly triad, algorithms such as gradient TD-learning (GTD) [1]}, TD-learning with gradient correction (TDC) [2]}, and emphatic TD-learning [3]} all require to maintain two iterates. Such two time-scale algorithms are in general harder to implement, and in addition, even if convergence is guaranteed, there is no characterization on the limit point. By using multi-step TD-learning, Algorithm REF  naturally achieves convergence, requires to maintain only one iterate, and has a limit point that can be characterized as the solution of a projected Bellman equation.
