We assume that students are trained using an actor-critic algorithm such as PPO [1]} where a value function critic \(V(s_t)\)  predicts future returns. In our implementation the target \(\hat{V}(s_t)\)  for the value function critic is implemented using General Advantage Estimation (GAE) [2]}.
