Learning a control policy for a text-adventure game requires a significant amount of exploration, resulting in training runs that take hundreds of thousands of simulations (Narasimhan et al., 2015;<cite> Ammanabrolu and Riedl, 2019)</cite> .