The BBQ model is updated as follows. Given observed transitions \(\mathcal {H} = \lbrace (s, a, r, s^{\prime })\rbrace \) , one uses the target network to compute the target values \(y\)  for each \((s,a)\)  in \(\mathcal {H}\) , resulting in the set \(\mathcal {D} = \lbrace (x,y)\rbrace \) , where \(x=(s,a)\)  and \(y\)  may be computed as in DQN [1]}. Then, parameter \(\theta \)  is updated to represent the posterior distribution of weights. Since the exact posterior is not Gaussian any more, and thus not representable by BBQ, it is approximated as follows: \(\theta \)  is chosen by minimizing the variational free energy [2]}, the KL-divergence between the variational approximation \(q(\mathbf {w}|\theta )\) 
and the posterior \(p(\mathbf {w}|\mathcal {D})\) :
\(\theta ^* &=& \operatorname{argmin}_{\theta } \operatorname{KL}[q(\mathbf {w}|\theta ) || p(\mathbf {w}| \mathcal {D})] \\&=& \operatorname{argmin}_{\theta } \Big \lbrace  \operatorname{KL}[q(\mathbf {w}|\theta ) || p(\mathbf {w})]- \mathbf {E}_{q(\mathbf {w}|\theta )}[\log p(\mathcal {D}|\mathbf {w})] \Big \rbrace  \,.\) 
