Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel. We introduce a new sender-receiver game to study emergent communication for this spectrum of partially-competitive scenarios and put special care into evaluation. We find that communication can indeed emerge in partially-competitive scenarios, and we discover three things that are tied to improving it. First, that selfish communication is proportional to cooperation, and it naturally occurs for situations that are more cooperative than competitive. Second, that stability and performance are improved by using LOLA (Foerster et al, 2018), especially in more competitive scenarios. And third, that discrete protocols lend themselves better to learning cooperative communication than continuous ones. First and foremost, we show evidence against the current notion that selfish agents do not learn to communicate, and we hope our findings encourage more research into communication under The comparison between discrete and continuous communication for both the REINFORCE-deterministic setup as well as 1-step LOLA agents is shown in Figure 4a . We see that though overall continuous communication can achieve highest information transfer, the gains in performance seem to mostly from manipulation of the sender by the receiver. Two examples are shown for REINFORCE agents in Figures 4b,4c . To find a trend, we plot all 100 hyperparameter runs for b ∈ [3, 6, 9, 12] between continuous and discrete communication using 1-step LOLA agents in Figures 4d,4e ,4f,4g. We find that manipulation is the common result in continuous communication though individual cooperative points can sometimes be found. In general, continuous communication does not lend itself to cooperative communication competition. We have shown three important properties of communication. First, a game being more cooperative than competitive is sufficient to naturally emerge communication. Second, we've clarified the distinction between information transfer, communication, and manipulation, providing motivation for a better quantitative metric to measure emergent communication in competitive environments. Next, we've found that LOLA improves effective selfish communication and, using our metric, we find it does so by improving both agents' performance and stability. Finally, we've shown that using a discrete communication channel encourages the learning of cooperative commu-nication in contrast to the continuous communication channel setting, where we find little evidence of cooperation. In fully-cooperative emergent communication, both agents fully trust each other, so cooperatively learning a protocol is mutually beneficial. In competitive MARL, the task is using an existing protocol (or action space) to compete with each other. However, selfish emergent communication combines these two since the inherent competitiveness of using the protocol to win is tempered by the inherent cooperativeness of learning it; without somewhat agreeing to meanings, agents cannot use those meanings to compete (Searcy & Nowicki, 2005; Skyrms & Barrett, 2018) . Thus, the agents must both learn a protocol and use that protocol simultaneously. In this way, even while competing, selfish agents emerging a communication protocol must learn to cooperate.
We manage to emerge communication with selfish agents, contrary to the current view in ML
{"src": "scitldr-AIC_train", "paper_id": "B1liIlBKvS"}