Our algorithm is trained for 1000 epochs, where each epoch consists of 1000 mini-batch gradient descent steps.
After each epoch of training, a rollout of 10 episodes is performed.
Both our algorithm and baselines are run under five random seeds \(\left\lbrace 0,1,2,3,4\right\rbrace \) , and we report the mean and standard deviation of the final rollouts across the five seeds.
We follow [1]} to rerun the baselines under the recommended hyperparameter setting, including dataset-specific hyperparameters if available.
