Modern virtual personal assistants provide a convenient interface for completing daily tasks via voice commands. An important consideration for these assistants is the ability to recover from automatic speech recognition (ASR) and natural language understanding (NLU) errors. In this paper, we focus on learning robust dialog policies to recover from these errors. To this end, we develop a user simulator which interacts with the assistant through voice commands in realistic scenarios with noisy audio, and use it to learn dialog policies through deep reinforcement learning. We show that dialogs generated by our simulator are indistinguishable from human generated dialogs, as determined by human evaluators. Furthermore, preliminary experimental results show that the learned policies in noisy environments achieve the same execution success rate with fewer dialog turns compared to fixed rule-based policies.
Given that the dialog state is not fully observable due to ASR NLU errors and uncertainty over a user's goals, researchers have also investigated the use of Partially Observable Markov Decision Processes (POMDPs) @cite_19 . However, these systems are complex and exact policy learning for POMDPs is intractable @cite_19 . Rule-based policies, on the other hand, use ASR NLU confidence scores to recover from errors. However, asking too many clarifying questions can frustrate users and in some scenarios it might be more beneficial to execute on available information. Additionally, ASR NLU components are not static and as such policies with fixed confidence levels need to be constantly updated.
@cite_19
1975244201
Statistical dialog systems (SDSs) are motivated by the need for a data-driven framework that reduces the cost of laboriously handcrafting complex dialog managers and that provides robustness against the errors created by speech recognizers operating in noisy environments. By including an explicit Bayesian model of uncertainty and by optimizing the policy via a reward-driven process, partially observable Markov decision processes (POMDPs) provide such a framework. However, exact model representation and optimization is computationally intractable. Hence, the practical application of POMDP-based systems requires efficient algorithms and carefully constructed approximations. This review article provides an overview of the current state of the art in the development of POMDP-based spoken dialog systems.