More fundamental is the learning algorithm almost universally employed in deep RL research: backpropagation. As has been widely discussed in connection with supervised deep learning research, which also uses backpropagation, there are outstanding questions about how backpropagation might be implemented in biological neural systems, if indeed it is at all [1]}, [2]} [3]}, [4]}. And there are inherent difficulties within backpropagation associated with preserving the results of old learning in the face of new learning, a problem for which remedies are being actively researched, in some cases taking inspiration from neuroscience [5]}.
