After a C-KBQA system is deployed to serve users, there may be a need over time to add more data types (new tables or entity attributes) to the KB. This problem, referred to as domain extension [1]}, makes exploration even more challenging: the system needs to explicitly quantify the uncertainty in its parameters for attributes/tables, so as to explore new ones more aggressively while avoiding exploring those that have already been learned. [2]} approach the problem using a Bayesian-by-Backprop variant of Deep Q-Network (DQN) [3]}.
DQN learns a neural-network-based Q-function \(Q(s,a;\theta )\)  parameterized by \(\theta \) . Since Q-function measures the maximum expected reward for each \((s, a)\)  pair, if the optimal Q-function is available, the optimal policy can be determined by \(\pi (s;\theta ) = \text{argmax}_a Q(s,a;\theta )\) .
