One of the distinguishing aspects of human language is its compositionality, which allows us to describe complex environments with limited vocabulary. Previously, it has been shown that neural network agents can learn to communicate in a highly structured, possibly compositional language based on disentangled input (e.g. hand- engineered features). Humans, however, do not learn to communicate based on well-summarized features. In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols. The agents play an image description game where the image contains factors such as colors and shapes. We train the agents using the obverter technique where an agent introspects to generate messages that maximize its own understanding. Through qualitative analysis, visualization and a zero-shot test, we show that the agents can develop, out of raw image pixels, a language with compositional properties, given a proper pressure from the environment. One of the key requirements for artificial general intelligence (AGI) to thrive in the real world is its ability to communicate with humans in natural language. Natural language processing (NLP) has been an active field of research for a long time, and the introduction of deep learning BID18 enabled great progress in NLP tasks such as translation, image captioning, text generation and visual question answering Vinyals et al., 2015; BID13 BID10 Serban et al., 2016; BID19 BID0 . However, training machines in a supervised manner with a large dataset has its limits when it comes to communication. Supervised methods are effective for capturing statistical associations between discrete symbols (i.e. words, letters). The essence of communication is more than just predicting the most likely word to come next; it is a means to coordinate with others and potentially achieve a common goal BID1 BID7 Wittgenstein, 1953 ).An alternative path to teaching machines the art of communication is to give them a specific task and encourage them to learn how to communicate on their own. This approach will encourage the agents to use languages grounded to task-related entities as well as communicate with other agents, which is one of the ways humans learn to communicate BID5 . Recently , there have been several notable works that demonstrated the emergence of communication between neural network agents. Even though each work produced very interesting results of its own, in all cases, communication was either achieved with a single discrete symbol (as opposed to a sequence of discrete symbols) BID8 BID17 or via a continuous value (Sukhbaatar et al., 2016; BID12 . Not only is human communication un-differentiable, but also using a single discrete symbol is quite far from natural language communication. One of the key features of human language is its compositional nature; the meaning of a complex expression is determined by its structure and the meanings of its constituents BID9 . More recently , BID22 and BID16 trained the agents to communicate in grounded, compositional language. In both studies , however, inputs given to the agents were hand-engineered features (disentangled input) rather than raw perceptual signals that we receive as humans.In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols. Unlike previous works, our setup poses greater challenges to the agents since visual understanding and discrete communication have to be induced from scratch in parallel. We place the agents in a two-person image description game, where images contain objects of various color and shape. Inspired by the pioneering work of BID3 , we employ a communication philosophy named obverter to train the agents. Having its root in the theory of mind (Premack & Woodruff, 1978) and human language development BID21 , the obverter technique motivates an agent to search over messages and generate the ones that maximize their own understanding. The contribution of our work can be summarized as follows:• We train artificial agents to learn to disentangle raw image pixels and communicate in compositional language at the same time.• We describe how the obverter technique, a differentiable learning algorithm for discrete communication, could be employed in a communication game with raw visual input.• We visualize how the agents are perceiving the images and show that they learn to disentangle color and shape without any explicit supervision other than the communication one.• Experiment results suggest that the agents could develop, out of raw image input, a language with compositional properties, given a proper pressure from the environment (i.e. the image description game).Finally, while our exposition follows a multi-agent perspective, it is also possible to interpret our results in the single-agent setting. We are effectively learning a neural network that is able to learn disentangled compositional representations of visual scenes, without any supervision. Subject to the constraints imposed by their environment, our agents learn disentangled concepts, and how to compose these to form new concepts. This is an important milestone in the path to AGI. In this work, we used the obverter technique to train neural network agents to communicate in a two-person image description game. Through qualitative analysis, visualization and the zero-shot test, we have shown that even though the agents receive raw perception in the form of image pixels, under the right environment pressures, the emerged language had properties consistent with the ones found in compositional languages.As an evaluation strategy, we followed previous works and focused on assessing the necessary conditions of compositional languages. However, the exact definition of compositional language is still somewhat debatable, and, to the best of our knowledge, there is no reliable way to mathematically quantify the degree of compositionality of an arbitrary language. Therefore, in order to encourage active research and discussion among researchers in this domain, we propose for future work, a quantitatively measurable definition of compositionality. We believe compositionality of a language is not binary (e.g. language A is compositional/not compositional), but a spectrum. For example, human language has some aspects that are compositional (e.g., syntactic constructions, most morphological combinations) and some that are not (e.g., irregular verb tenses in English, character-level word composition). It is also important to clearly define grounded language and compositional language. If one agent says abc (eat red apple) and another says cba (apple red eat), and they both understand each other, are they speaking compositional language? We believe such questions should be asked and addressed to shape the definition of compositionality.In addition to the definition/evaluation of compositional languages, there are numerous directions of future work. Observing the emergence of a compositional language among more than two agents is an apparent next step. Designing an environment to motivate the agents to disentangle more than two factors is also an interesting direction. Training agents to consider the context (i.e. pragmatics), such as giving each agent several images instead of one, is another exciting future work. A EMERGENCE OF GRAMMAR, BID3 In BID3 , the author successfully trained neural agents to develop a structured (i.e. grammatical) language using disentangled meaning vectors as the input. Using 10 subject vectors and 10 predicate vectors, all represented as explicit binary vectors, total 100 meaning vectors could be composed TAB7 ). Each digit in the subject vector 5a serves a clear role, respectively representing speaker(sp), hearer(hr), other(ot), and plural(pl). The predicate vector values, on the other hand, are randomly chosen so that each predicate vector will have three 1's and three 0's. The combination of ten subject vectors and ten predicate vectors allows 100 meaning vectors.The author used twenty neural agents for the experiment. Each agent was implemented with the vanilla recurrent neural networks (RNN), where the hidden vector h's size was 10, same as the size of the meaning vector m in order to treat h as the agent's understanding of m. In each training round a single learner (i.e. listener) and ten teachers (i.e. speaker) were randomly chosen. Each teacher, given all 100 m's in random order, generates a message s 5 for each m and sends it to the learner. The messages are generated using the obverter techinque, which is described in Algorithm 1. The learner is trained to minimize the mean squared error (MSE) between h (after consuming the s) and m. After the learner has learned from all ten teachers, the next round begins, repeating the process until the error goes below some threshold.Algorithm 1: Message generation process used in BID3 . DISPLAYFORM0 9 Append i to s; DISPLAYFORM1 Terminate;When the training was complete, the author was able to find strong patterns in the messages used by the agents ( Table 6 ). Note that the messages using predicates tired, scared, sick and happy especially follow a very clear pattern. Batali also conducted a zero-shot test where the agents were trained without the diagonal elements in Table 6 and tested with all 100 meaning vectors. The agents were able to successfully communicate even when held-out meaning vectors were used, but the Table 6 : (Top) Messages used by a majority of the population for each of the given meanings.(Bottom ) A potential analysis of the system in terms of a root plus modifications. Italic symbols are used to specify predicates and roman symbols are used to specify subjects. Messages in parentheses cannot be made to fit into this analysis.messages used for the held-out meaning vectors did not show as strong compositional patterns as the non-zero-shot case.
We train neural network agents to develop a language with compositional properties from raw pixel input.
{"src": "scitldr-AIC_train", "paper_id": "rknt2Be0-"}