Chat as Expected: Learning to Manipulate
Black-box Neural Dialogue Models
Haochen Liu
Michigan State University
liuhaoc1@msu.edu
Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
Recently, neural network based dialogue systems have become ubiquitous in our
increasingly digitalized society. However, due to their inherent opaqueness,
some recently raised concerns about using neural models are starting to be
taken seriously. In fact, intentional or unintentional behaviors could lead to
a dialogue system to generate inappropriate responses. Thus, in this paper, we
investigate whether we can learn to craft input sentences that result in a
black-box neural dialogue model being manipulated into having its outputs
contain target words or match target sentences. We propose a reinforcement
learning based model that can generate such desired inputs automatically.
Extensive experiments on a popular well-trained state-of-the-art neural
dialogue model show that our method can successfully seek out desired inputs
that lead to the target outputs in a considerable portion of cases.
Consequently, our work reveals the potential of neural dialogue models to be
manipulated, which inspires and opens the door towards developing strategies to
defend them.
Haochen Liu Zhiwei Wang Tyler Derr Jiliang Tang