When communicating, humans rely on internally-consistent language representations. That is, as speakers, we expect listeners to behave the same way we do when we listen. This work proposes several methods for encouraging such internal consistency in dialog agents in an emergent communication setting. We consider two hypotheses about the effect of internal-consistency constraints: 1) that they improve agents’ ability to refer to unseen referents, and 2) that they improve agents’ ability to generalize across communicative roles (e.g. performing as a speaker de- spite only being trained as a listener). While we do not find evidence in favor of the former, our results show significant support for the latter. Emergent communication is the study of how linguistic protocols evolve when agents are tasked to cooperate. For example, agents engaged in a simple object retrieval task learn to communicate with one another in order to get the items they want . To date, work of this type has each agent assume a conversational role. Thus, agents are often trained only to speak or only to listen , or similarily trained to speak using a vocabulary disjoint from the vocabulary it is understands as a listener-e.g. speaking only to ask questions ("what color?") and listening only to comprehend the answer ("blue") Das et al., 2017) . These assumptions are misaligned with how we think about human communication, and with the way we'd like computational models to work in practice. As humans, not only can we easily shift between roles, we also know that there is inherent symmetry between these roles: we expect others to speak (or listen) similarly to the way we do, and we know that others expect the same of us. We test if dialog agents that incorporate the symmetry between themselves and their communicative partners learn more generalizable representations than those which do not. We introduce three modifications to the agents to encourage that they abide by the "golden rule": speak/listen as you would want to be spoken/listened to. Specifically, these modifications include self-play training objectives, shared embedding spaces, and symmetric decoding and encoding mechanisms that share parameters. We test two hypotheses about the effect of the proposed modifications on emergent communication: 1. Internal-consistency constraints improve agents' ability to generalize to unseen items-e.g. training on "red square" and "blue circle" and then testing on "blue square". 2. Internal-consistency constraints improve agents' ability to generalize across communicative roles-e.g. training on "blue" as a listener, and using "blue" as a speaker when testing. We evaluate the effect of each of the proposed modifications with two reference game datasets and two model architectures, an RNN model used by and a Transformer model. We find no evidence to support that internal-consistency improves generalization to unseen items (Hypothesis 1), but significant evidence that these proposed constraints enable models to generalize learned representations across communicative roles (Hypothesis 2), even in the case of where the agent receives no direct training in the target (test) role. All of our code and data are available at bit.ly/internal-consistency-emergent-communication. Notation. The space of possible references is parameterized by the number of attributes n f that describe each item (e.g. color) and the number of values n v each attribute can take (e.g.{red, blue}). Each item o is a bag-of-features vector o P t0, 1u N where N " n f¨nv . Each index o i is 1 if o expresses the ith feature value. The speaker produces a message with symbols from a vocabulary V with length L. For comparison, we use the best-performing setting |V| " 100 and L " 10 from previous work . Symbols in V are represented as 1-hot vectors. In each round of the reference game, we construct xC, r, ry where C is the context (set of item column vectors stacked into a matrix), r is a vector representing the referent, and r is the index of the referent in C. We uniformly sample k´1 items as distractors to form C " to 1 , . . . o k´1 uYtru. The distractors are is sampled randomly each round (in every epoch). We propose three methods for encouraging dialog agents to follow "the golden rule": speak/listen to others as you would expect to be spoken/listened to. In the emergent communication setting, we find that the internal-consistency constraints do not systematically improve models' generalization to novel items, but both the self-play objective and shared embeddings significantly improve performance when agents are tested on roles they were not directly trained for. In fact, when trained in one role and tested on another, these internal-consistency constraints allow the agents to perform about as well as if they had been trained in the target role.
Internal-consistency constraints improve agents ability to develop emergent protocols that generalize across communicative roles.
{"src": "scitldr-AIC_train", "paper_id": "SkgJOAEtvr"}