The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by using contemporary deep learning methods and by training reinforcement-learning neural network agents on referential communication games. We extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. We find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured.   The study of emergent communication is important for two related problems in language development, both human and artificial: language evolution, the development of communication protocols from scratch BID27 ; and language acquisition, the ability of an embodied agent to learn an existing language. In this paper we focus on the problem of how environmental or pre-linguistic conditions affect the nature of the communication protocol that an agent learns. The increasing realism and complexity of environments being used for grounded language learning BID8 BID18 present an opportunity to analyse these effects in detail.In line with previous work on emergent communication, we are strongly motivated by the view that language derives meaning from its use BID39 BID37 . This perspective especially motivates the study of language emergence in cases where co-operative agents try to achieve shared goals in game scenarios BID34 BID6 BID26 , and is related to the study of multi-agent and self-play methods that have found great success in other areas of machine learning BID1 BID30 . Here we focus on simple referential games, in which one agent must communicate to another a target object in the agent's environment.One of the most important properties of natural language is compositionality. Smaller building blocks (e.g. words, morphemes) are used to generate unbounded numbers of more complex forms (e.g. sentences, multi-word expressions), with the meaning of the larger form being determined by the meanings of its parts and how they are put together BID14 . Compositionality is an advantage in any communication protocol as it allows in principle infinite expression through a finite dictionary and a finite set of combination rules. In emergent communication research, previous work has shown that agents can produce (somewhat) compositional protocols when engaging in language games BID34 . However, the computational agents were typically situated in artificial worlds containing just a handful of objects, represented as disentangled, structured, and sometimes even atomic symbols, e.g. attribute-based or one-hot vectors BID2 BID5 BID13 BID0 BID26 . However, humans receive raw sensorimotor hank you! h a n k y o u ! rather than symbolic input, and little work to date has tested whether these findings carry over when agents are situated in less idealized worlds that bear more similarity to the kind of entangled and noisy environments to which humans are typically exposed. We presented a series of studies investigating the properties of protocols emerging when reinforcement learning agents are trained end-to-end on referential communication games. We found that when agents are presented with disentangled input data in the form of attribute vectors, this inherent compositional structure is successfully retained in the output. Moreover, we showed that communication can also be achieved in cases where agents are presented with raw pixel data, a type of input that aligns better with the raw sensorimotor data that humans are exposed to. At the same time, we found that their ability to form compositional protocols in these cases is hampered by their ability to pull apart the objects' factors of variations. Altogether, we were able to successfully scale up traditional research from the language evolution literature on emergent communication tasks to the contemporary deep learning framework, thus opening avenues to more realistic, and large scale, computational simulations of language emergence with complex image stimuli.
A controlled study of the role of environments with respect to properties in emergent communication protocols.
{"src": "scitldr-AIC_train", "paper_id": "HJGv1Z-AW"}