Any distribution over complete trajectories that arises from a forward policy satisfies a Markov property: the marginal choice of action out of a state \(s\)  is independent of how \(s\)  was reached. Conversely, any Markovian distribution over \(\mathcal {T}\)  arises from a forward policy [1]}.
