The human ability to coordinate and cooperate has been vital to the development of societies for thousands of years. While it is not fully clear how this behavior arises, social norms are thought to be a key factor in this development. In contrast to laws set by authorities, norms tend to evolve in a bottom-up manner from interactions between members of a society. While much behavior can be explained through the use of social norms, it is difficult to measure the extent to which they shape society as well as how they are affected by other societ al dynamics. In this paper, we discuss the design and evaluation of a reinforcement learning model for understanding how the opportunity to choose who you interact with in a society affects the overall societ al outcome and the strength of social norms. We first study the emergence of norms and then the emergence of cooperation in presence of norms. In our model, agents interact with other agents in a society in the form of repeated matrix-games: coordination games and cooperation games. In particular, in our model, at each each stage, agents are either able to choose a partner to interact with or are forced to interact at random and learn using policy gradients.
Behavioral norms are key ingredients that allow agent coordination where societ al laws do not sufficiently constrain agent behaviors. Whereas social laws need to be enforced in a top-down manner, norms evolve in a bottom-up manner and are typically more self-enforcing. While effective norms can significantly enhance performance of individual agents and agent societies, there has been little work in multiagent systems on the formation of social norms. We propose a model that supports the emergence of social norms via learning from interaction experiences. In our model, individual agents repeatedly interact with other agents in the society over instances of a given scenario. Each interaction is framed as a stage game. An agent learns its policy to play the game over repeated interactions with multiple agents. We term this mode of learning social learning, which is distinct from an agent learning from repeated interactions against the same player. We are particularly interested in situations where multiple action combinations yield the same optimal payoff. The key research question is to find out if the entire population learns to converge to a consistent norm. In addition to studying such emergence of social norms among homogeneous learners via social learning, we study the effects of heterogeneous learners, population size, multiple social groups, etc. Social norms play a pivotal role in sustaining social order by regulating individual behaviors in a society. In normative multiagent systems, social norms have been used as an efficient mechanism to govern virtual agent societies towards cooperation and coordination. In this paper, we study the emergence of social norms via learning from repeated local interactions in networked agent societies. We propose a collective learning framework, which imitates the opinion aggregation process in human decision making, to study the impact of agent local collective behaviors on norm emergence in different situations. In the framework, each agent interacts repeatedly with all of its neighbors. At each step, an agent first takes a best-response action towards each of its neighbors and then combines all of these actions into a final action using ensemble learning methods. We conduct extensive experiments to evaluate the framework with respect to different network topologies, learning strategies, numbers of actions, and so on. Experimental results reveal some significant insights into norm emergence in networked agent societies achieved through local collective behaviors. A social norm is a behavior that emerges as a convention within society without any direction from a central authority. Social norms emerge as repeated interactions between individuals give rise to biases toward actions or behaviors which spread through the society until one behavior is adapted as the default behavior, even when multiple acceptable behaviors exist. Of particular interest to us is how and when norms emerge in social networks, which provide a framework for individuals to interact routinely. We study how quickly norms converge in social networks depending on parameters such as the topology of the network, population size, neighborhood size, and number of behavior alternatives. Our research can be used to model and analyze popular social networks on the Internet such as Facebook, Flickr, and Digg. In addition, it can be used to predict how norms emerge and spread in human societies, ranging from routine decisions like which side of the road to drive on to social trends such as the green phenomenon. Abstract Learning to act in a multiagent environment is a difficult problem since the normal definition of an optimal policy no longer applies. The optimal policy at any moment depends on the policies of the other agents. This creates a situation of learning a moving target. Previous learning algorithms have one of two shortcomings depending on their approach. They either converge to a policy that may not be optimal against the specific opponents' policies, or they may not converge at all. In this article we examine this learning problem in the framework of stochastic games. We look at a number of previous learning algorithms showing how they fail at one of the above criteria. We then contribute a new reinforcement learning technique using a variable learning rate to overcome these shortcomings. Specifically, we introduce the WoLF principle, “Win or Learn Fast”, for varying the learning rate. We examine this technique theoretically, proving convergence in self-play on a restricted class of iterated matrix games. We also present empirical results on a variety of more general stochastic games, in situations of self-play and otherwise, demonstrating the wide applicability of this method.