A straightforward way to generalize the problem setting for meta-RL is in terms of an underspecified partially observable Markov decision problem [1]}. This enriches the standard POMDP tuple \(\langle S, A, \Omega , T, R, O\rangle \) , respectively a set of states, actions, and observations, together with state-transition, reward and observation functions [2]}, adding a set of parameters \(\Theta \)  which govern the latter three functions. Importantly, \(\Theta \)  is understood as a random variable, governed by a prior distribution and resulting in a corresponding distribution of POMDPs. In this setting, meta-RL can be viewed in terms of hierarchical Bayesian inference, with a relatively slow process, spanning samples, gradually inferring the structure of the parameterization \(\Theta \) , and in turn supporting a rapid process which infers the specific parameters underlying each new draw from the task distribution [3]}, [4]}, [5]}, [6]}. In meta-RL, the latter process is an active one, involving strategic gathering of information or experimentation [7]}, [8]}.
