Despite much promise, Deep RL does not always provide effective end-to-end planning performance. Specifically, RL can struggle when there are sparse rewards [1]} or on long horizon tasks [2]}. In addition, achieving convergence often requires significant hyper-parameter tuning and reward shaping [3]}. As a result, many past works have used Deep RL to solve a smaller part of the planning problem. For example, [4]} used Deep RL to estimate reachability, [1]} used RL for local planning, and [6]}, [7]}, [8]} used RL to learn high-level actions (primitives).
