Similar to [3, <cite>13</cite>, 30] , we formulate this problem as a Markov Decision Process (MDP), in which the goal is to train a policy gradient agent (using REIN- FORCE [28] ) to learn an optimal reasoning path to answer a given query (e s , r, ?).