This perspective brings into view two further desiderata for any benchmark meta-RL task distribution. First, the ground-truth parameterization of the distribution should ideally be accessible. This allows agent performance to be compared directly against an optimal baseline, which is precisely a Bayesian learner, sometimes referred to as an `ideal observer' [1]}, [2]}. Second, the structure of the task distribution should be interesting, in that it displays properties comparable to those involved in many challenging real-world tasks. Intuitively, in the limit, interesting structure should feature compositionality, causal relationships, and opportunities for conceptual abstraction [3]}, and result in tasks whose diagnosis and solutions require strategic sequencing of actions.
<FIGURE>