We have shown that this simple mechanism for generating task-level explanations can be used to provide explanations to any RL algorithm. The explanations generated by CTDL consist of a simple list of states, actions and values, which form the basis of most Deep RL algorithms. For a Deep RL algorithm to use the list during learning, all it requires is a kernel function that decides when to use an entry from the list. In the present work we used the Euclidean distance between the current state and the states in the list to make this decision based on a pre-defined threshold. However, this kernel function could take other potentially more useful forms. One interesting alternative is the Successor Representation (SR) [1]}, [2]}, which calculates future state occupancies based on the current state. Based on the SR, if a state in the list is highly likely to be visited in the near future then the agent should use the associated value and action form the explanation. Future work should therefore explore the utility of different kernel functions for deciding when to use the contents of the explanation.
