Taming Continuous Posteriors for Latent Variational Dialogue Policies
Utilizing amortized variational inference for latent-action reinforcement
learning (RL) has been shown to be an effective approach in Task-oriented
Dialogue (ToD) systems for optimizing dialogue success. Until now, categorical
posteriors have been argued to be one of the main drivers of performance. In
this work we revisit Gaussian variational posteriors for latent-action RL and
show that they can yield even better performance than categoricals. We achieve
this by simplifying the training procedure and propose ways to regularize the
latent dialogue policy to retain good response coherence. Using continuous
latent representations our model achieves state of the art dialogue success
rate on the MultiWOZ benchmark, and also compares well to categorical latent
methods in response coherence.