The model proposed in [1]}, called BBQ, is identical to DQN, except that it maintains a posterior distribution \(q\)  over the network weights \(\mathbf {w}=(w_1,\ldots ,w_N)\) . For computational convenience, \(q\)  is a multivariate Gaussian distribution with diagonal covariance, parameterized by \(\theta =\lbrace (\mu _i,\rho _i)\rbrace _{i=1}^N\) , where weight \(w_i\)  has a Gaussian posterior distribution, \(\mathcal {N}(\mu _i, \sigma _i^2)\)  and \(\sigma _i = \log (1+\exp (\rho _i))\) . The posterior information leads to a natural exploration strategy, inspired by Thompson Sampling [2]}, [3]}, [4]}. When selecting actions, the agent simply draws a random weight \(\tilde{\mathbf {w}} \sim q\) , and then selects the action with the highest value output by the network. Experiments show that BBQ explores more efficiently than state-of-the-art baselines for dialogue domain extension.
