Automatic optimization of spoken dialog management policies that are robust to environmental noise has long been the goal for both academia and industry. Approaches based on reinforcement learning have been proved to be effective. However, the numerical representation of dialog policy is human-incomprehensible and difficult for dialog system designers to verify or modify, which limits its practical application. In this paper we propose a novel framework for optimizing dialog policies specified in domain language using genetic algorithm. The human-interpretable representation of policy makes the method suitable for practical employment. We present learning algorithms using user simulation and real human-machine dialogs respectively. Empirical experimental results are given to show the effectiveness of the proposed approach.
proposed a hybrid learning method in @cite_15 to learn a policy on an existing dialog corpus by combining the results of supervised and reinforcement learning. Pure RL on fixed dataset often shows irregular behavior due to the insufficient exploration problem. Supervised learning (SL) is used to mitigate the problem and the hybrid method shows better performance than pure SL or RL. In this regard the QVal fitness function is similar in spirit and the use of policy template can further constrain the DM behavior, thus is suitable for off-line on-corpus learning.
@cite_15
2152342063
We use single-agent and multi-agent Reinforcement Learning (RL) for learning dialogue policies in a resource allocation negotiation scenario. Two agents learn concurrently by interacting with each other without any need for simulated users (SUs) to train against or corpora to learn from. In particular, we compare the Qlearning, Policy Hill-Climbing (PHC) and Win or Learn Fast Policy Hill-Climbing (PHC-WoLF) algorithms, varying the scenario complexity (state space size), the number of training episodes, the learning rate, and the exploration rate. Our results show that generally Q-learning fails to converge whereas PHC and PHC-WoLF always converge and perform similarly. We also show that very high gradually decreasing exploration rates are required for convergence. We conclude that multiagent RL of dialogue policies is a promising alternative to using single-agent RL and SUs or learning directly from corpora.