While augmenting the agent's inputs is one way to impact its representation of current visual inputs, the recent deep RL literature suggests a different method for enriching internal representations, which is to add auxiliary tasks to the agent's training objective [1]}. As one application of this idea, we added to the RL objective a set of supervised tasks, the objectives of which were to produce outputs indicating (1) the total number of stones present in each of a set of perceptual categories, (2) the total number of potions present of each color, and (3) the currently prevailing ground-truth chemistry (see Appendix REF  for further details of this augmentation).Prediction tasks (1) and (2) were always done in conjunction and are collectively referred to as `Predict: Features', while (3) is referred to as `Predict: Chemistry'.
<FIGURE>