DOI: 10.1017/S000000000000000 Printed in the United Kingdom A Survey of Statistical User Simulation Techniques for Reinforcement-Learning of Dialogue Management Strategies <s> Dialogue Management Strategies <s> We present results from an extensive empirical analysis of non-understanding errors and ten non-understanding recovery strategies, based on a corpus of dialogs collected with a spoken dialog system that handles conference room reservations. More specifically, the issues we investigate are: what are the main sources of non-understanding errors? What is the impact of these errors on global performance? How do various strategies for recovery from non-understandings compare to each other? What are the relationships between these strategies and subsequent user response types, and which response types are more likely to lead to successful recovery? Can dialog performance be improved by using a smarter policy for engaging the non-understanding recovery strategies? If so, can we learn such a policy from data? Whenever available, we compare and contrast our results with other studies in the literature. Finally, we summarize the lessons learned and present our plans for future work inspired by this analysis. <s> BIB001 </s> DOI: 10.1017/S000000000000000 Printed in the United Kingdom A Survey of Statistical User Simulation Techniques for Reinforcement-Learning of Dialogue Management Strategies <s> Dialogue Management Strategies <s> We propose a novel approach for constructing more accurate beliefs over concept values in spoken dialog systems by integrating information across multiple turns in the conversation. In particular, we focus our attention on updating the confidence score of the top hypothesis for a concept, in light of subsequent user responses to system confirmation actions. Our data-driven approach bridges previous work in confidence annotation and correction detection, providing a unified framework for belief updating. The approach significantly outperforms heuristic rules currently used in most spoken dialog systems <s> BIB002
In order to complete the tasks described above and to decide "what to say", the dialogue manager needs to track the dialogue history and update some representation of the current state of the dialogue. In addition, the DM needs a dialogue strategy that defines the conversational behaviour of the system, for example when to take the initiative in a dialogue or when to establish common ground. The performance of a spoken dialogue system is highly dependent on the quality of its DM strategy. Unfortunately, the design of a good strategy is far from trivial since there is no clear definition as to what constitutes a good strategy. User populations are often diverse, thus making it difficult to foresee which form of system behaviour will lead to quick and successful completion of the dialogue. In addition, the omnipresence of recognition errors leads to constant uncertainty about the true intention of the user. As applications reach greater levels of complexity, the process of designing robust dialogue strategies becomes very time-consuming and expensive. To further illustrate the complexity involved in dialogue strategy design, the following subsections briefly discuss two frequently arising design issues: the degree of initiative a system can take in a dialogue and the choice of confirmation strategy. Our goal here is not to provide a complete overview or to argue in favour of any specific design choices, but to indicate the potential benefits of automatic machinelearning techniques in the context of dialogue strategy design. Two issues which have recently received much attention but are not covered here include error handling and recovery  BIB001 and accounting for uncertainty introduced by the noisy communication channel BIB002  Williams et al., 2005a,b) .