This can be achieved using reinforcement learning (RL), as described in Section REF . However, current RL methods require a large amount of user-system interaction data, which may have too high a cost in real-world settings.
To improve the sample efficiency in exploration, three main strategies are considered in the research community [1]}.
The first is to encourage the system to explore dialog states which are less frequently visited, and thus cause more uncertainty for the system in deciding how to respond [2]}, [3]}, [4]}, [5]}. The second, known as model-based RL, is to use an environment model for efficient exploration. In task-oriented dialog, the environment model is a user simulator, which can be used alone (Figure REF  (Middle)) or
together with human users (Figure REF  (Right)) for dialog policy learning.
The third strategy is decomposing dialog tasks into smaller subtasks or skills that are easier to learn [6]}, [7]}, [8]}, [9]}. Ideally, we want to identify a set of basic skills that are common across many tasks. After learning these basic skills, a dialog system can learn to complete more complex tasks more easily via hierarchical reinforcement learning [10]}.
