Given hyperparameter vector \(u\in \mathcal {U}\) , and task \(t_i\in \mathcal {T}\) ,
in
the conventional single-task PAC-Bayes setting [1]},
the base-learner assumes a prior distribution \(\mathrm {P}\)  over \(\mathcal {W}\) . By observing the training data \(\mathbf {Z}_i^{\mathrm {M}}\) , the base learner
updates the prior distribution to a data-dependent distribution referred as posterior distribution \(\mathrm {Q}_i\) . Having a new instance, the base learner randomly picks a model parameter \(w\in \mathcal {W}\)  according to \(\mathrm {Q}_i\) .
To have a guarantee that the performance of training loss for the picked \(w\)  holds with
high probability as the performance of per-task generalization loss, we bound generalization gap averaged over
the posterior distribution, i.e., \(\mathbb {E}_{W\sim \mathrm {Q}_i}[\Delta \mathrm {L}(W|\mathbf {Z}^{\mathrm {M}}_i,u,t_i)]\) .
