We present an algorithm based on temporal difference (TD) techniques for PP-CPT-RL.
TD techniques seek to learn value functions using episodes of experience.
An experience episode comprises a sequence of states, actions, and rewards when following a policy \(\pi \) .
The predicted values at any time-step is updated in a way to bring it closer to the prediction of the same quantity at the next time-step.
In order to support environments with large, possibly continuous state spaces, our approach is based on deep Q-learning [1]}, where Q-values are parameterized by a deep neural networkThis parameterization is analogous to the Q-table that is typically seen in Q-learning [2]}. Updating entries of the Q-table is then equivalent to updating the values of the parameter \(\theta \)  using a gradient-based method..
