Reinforcement Learning. Value-based methods in RL use the Q-value function to choose the best action in discrete settings [1]}, [2]}. However, the Q-value approach cannot scale well to continuous environments. Policy Optimization techniques solve this by directly optimizing the policy, either learning online, using samples collected from the policy [3]}, [4]}, or offline, reusing the experience stored in a replay buffer [5]}, [6]}.
