Actions are generated according to some policy \(\pi \) , which may be deterministic, \(a = \pi (s)\) , or stochastic, \(a \sim \pi (a|s)\)  in which case \(\pi (a|s)\)  represents the probability of choosing action \(a\)  when the state is \(s\) . For any MDP there is an optimal policy which is deterministic [1]}. Nonetheless, defining the agent with a stochastic policy during training promotes exploration within the environment [2]}. The goal of an MDP is to design an optimal policy \(\pi ^* = \arg \max _\pi \operatornamewithlimits{\mathbb {E}}\left[ R_0 | \pi \right]\)  which maximizes Eq. (REF ) when actions are generated from that policy.
