In practice, we solve a sample-based approximation of the optimization problem in Equation REF  using a standard on-policy actor-critic procedure [1]}, [2]}. We rely on a dual gradient ascent procedure [3]} to ensure that the constraint is satisfied. Appendix  contains a complete description of the network architecture, choices of prior and encoding distribution, and the meta-learning procedure. For the purpose of this article, we are only interested in the properties of the fully-converged model and not in what happens during meta-learning.
