The first component of Equation REF  corresponds to the standard meta-reinforcement learning objective, while the second component ensures that the Kullback–Leibler (KL) divergence between a stochastic parameter encoding \(q(\mathbf {W} | \mathbf {\Lambda })\)  and a prior \(p(\mathbf {W})\)  remains smaller than some constant \(C\) . The KL term can be interpreted as the description length of neural network parameters, i.e., the number of bits required to store them.The desired coding length can, for example, be achieved using bits-back coding [1]} or minimal random coding [2]}. RL\(^3\)  therefore optimally trades-off performance against the number of bits required to implement the emerging reinforcement learning algorithm. It furthermore approximates the Bayes-optimal policy as \(C\)  goes to infinity under the same conditions as RL\(^2\)  (namely a sufficiently expressive function approximator and successful optimization to the global optimum). Note that the objective from Equation REF  can also be motivated by a PAC-Bayes bound on generalization performance to unseen tasks [3]}, [4]}, [5]}. While we focus on the resource-rational interpretation in the present article, we believe that both of these perspectives are complementary.
