We study the emergence of communication in multiagent adversarial settings inspired by the classic Imitation game. A class of three player games is used to explore how agents based on sequence to sequence (Seq2Seq) models can learn to communicate information in adversarial settings. We propose a modeling approach, an initial set of experiments and use signaling theory to support our analysis. In addition, we describe how we operationalize the learning process of actor-critic Seq2Seq based agents in these communicational games.
Attention has been recently brought to the communicational aspects of multiagent systems. Recent research has explored cooperative @cite_13 @cite_15 @cite_4 and semi-cooperative scenarios such as negotiation games @cite_5 . The emergence of communication in adversarial scenarios has been explored less extensively.
@cite_5 @cite_15 @cite_13 @cite_4
1764574858 1528676759 2153344215 2172933221
Fully cooperative multiagent systems--those in which agents share a joint utility model--is of special interest in AI. A key problem is that of ensuring that the actions of individual agents are coordinated, especially in settings where the agents are autonomous decision makers. We investigate approaches to learning coordinated strategies in stochastic domains where an agent's actions are not directly observable by others. Much recent work in game theory has adopted a Bayesian learning perspective to the more general problem of equilibrium selection, but tends to assume that actions can be observed. We discuss the special problems that arise when actions are not observable, including effects on rates of convergence, and the effect of action failure probabilities and asymmetries. We also use likelihood estimates as a means of generalizing fictitious play learning models in our setting. Finally, we propose the use of maximum likelihood as a means of removing strategies from consideration, with the aim of convergence to a conventional equilibrium, at which point learning and deliberation can cease. This exciting and pioneering new overview of multiagent systems, which are online systems composed of multiple interacting intelligent agents, i.e., online trading, offers a newly seen computer science perspective on multiagent systems, while integrating ideas from operations research, game theory, economics, logic, and even philosophy and linguistics. The authors emphasize foundations to create a broad and rigorous treatment of their subject, with thorough presentations of distributed problem solving, game theory, multiagent communication and learning, social choice, mechanism design, auctions, cooperative game theory, and modal logics of knowledge and belief. For each topic, basic concepts are introduced, examples are given, proofs of key results are offered, and algorithmic considerations are examined. An appendix covers background material in probability theory, classical logic, Markov decision processes and mathematical programming. Written by two of the leading researchers of this engaging field, this book will surely serve as THE reference for researchers in the fastest-growing area of computer science, and be used as a text for advanced undergraduate or graduate courses. We study a spectrum sharing problem in an unlicensed band where multiple systems coexist and interfere with each other. We first analyze a cooperative setting where all the systems collaborate to achieve a common goal. Under the assumptions that the systems communicate with Gaussian signals and treat interference as noise, we study the structure of the optimal power allocations. We show that any Pareto efficient vector of rates can be achieved with piece-wise constant power allocations. Moreover, if a strong interference condition among all the systems is satisfied, we show that frequency division multiplexing is optimal. We then consider a non-cooperative situation, where the systems act in a selfish and rational way, and investigate how the lack of cooperation can affect performance. Using game theory, we first analyze the possible outcomes of a one shot game, and observe that in many cases an inefficient solution results. We show that by extending the game definition to that of a repeated game, the possibility of building reputations and applying punishments allows to enlarge the set of achievable rates. We present examples that show that in many cases, the performance loss due to lack of cooperation is small. We also provide a converse theorem that proves that our results are tight and quantify the best achievable performance in a non-cooperative scenario Multiagent systems appear in most social, economical, and political situations. In the present work we extend the Deep Q-Learning Network architecture proposed by Google DeepMind to multiagent environments and investigate how two agents controlled by independent Deep Q-Networks interact in the classic videogame Pong. By manipulating the classical rewarding scheme of Pong we demonstrate how competitive and collaborative behaviors emerge. Competitive agents learn to play and score efficiently. Agents trained under collaborative rewarding schemes find an optimal strategy to keep the ball in the game as long as possible. We also describe the progression from competitive to collaborative behavior. The present work demonstrates that Deep Q-Networks can become a practical tool for studying the decentralized learning of multiagent systems living in highly complex environments.