15486536
Learning to commit in repeated games
Learning to converge to an efficient, i.e., Pareto-optimal Nash equilibrium of the repeated game is an open problem in multiagent learning. Our goal is to facilitate the learning of efficient outcomes in repeated plays of incomplete information games when only opponent's actions but not its payoffs are observable. We use a two-stage protocol that allows a player to unilaterally commit to an action, allowing the other player to choose an action knowing the action chosen by the committed player. The motivation behind commitment is to promote trust between the players and prevent them from mutually harmful choices made to preclude worst-case outcomes. Our agents learn whether commitment is beneficial or not. Interestingly, the decision to commit can be thought of as expanding the action space and our proposed protocol can be incorporated by any learning strategies used for playing repeated games. We show the improvement of the outcome efficiency of standard learning algorithms when using our proposed commitment protocol. We propose convergence to pareto optimal Nash equilibrium of repeated games as desirable learning outcomes. The performance evaluation in this paper uses a similarly motivated metric that measures the percentage of Nash equilibria for repated games that dominate the observed outcome.
12119095
Correlating Internal Parameters and External Performance: Learning Soccer Agents
We have developed a Soccer simulator in Java, which allows us to evaluate agent behavioral strategies for effectively collaborating with other team members while countering threats posed by the opposing team. Whereas action selection mechanisms to decide when to shoot, pass, dribble, guard opponent, tackle, etc. are important from a player's viewpoint, learning the capabilities of individual opponents from repeated encounters can provide critical information for the success of a team. Each agent is described by a set of skill levels for different soccer skills. A soccer agent uses its incomplete perceptions, model of the changing environment, knowledge of the skill levels of its own team players, and estimated skill levels of the opponent team players to select the most prudent action. In this paper, we identify learning opportunities for soccer agents, and investigate some of these possibilities in detail. Initial experimental results demonstrate the advantage of learning agents.
3526092
Selecting partners
1. I N T R O D U C T I O N In an open environment the goal of a self-interested agent would be to interact with or enter into partnership with those agents that will produce maximal local utility for this agent [5]. We assume that an agent can get one of several payoffs (utilities) for joining a particular coalition which is determined by a static probability distribution over the payoffs for each coalition. We consider choosing a single coalition to interact with repeatedly when the total number of interactions is known ahead of time.
43158386
Evolution and learning in multiagent systems
1062630
The Role of Commitment in Cooperative Negotiation
Cooperative information agents need mechanisms that enable them to work together effectively while solving common problems. We investigate the use of commitment by agents to proposed actions as a mechanism that allow agents to work concurrently on interdependent problems. Judicious use of commitment can not only increase the throughput of cooperative information systems, but also allow them to deal flexibly with dynamically changing environments. We use the domain of distributed scheduling to demonstrate that static commitment strategies are ineffective. Results from simulated experiments are used to identify the environmental features on which an adaptive commitment strategy should be predicated.
15974834
Learning Pareto-optimal Solutions in 2x2 Conflict Games
Multiagent learning literature has investigated iterated two-player games to develop mechanisms that allow agents to learn to converge on Nash Equilibrium strategy profiles. Such equilibrium configurations imply that no player has the motivation to unilaterally change its strategy. Often, in general sum games, a higher payoff can be obtained by both players if one chooses not to respond myopically to the other player. By developing mutual trust, agents can avoid immediate best responses that will lead to a Nash Equilibrium with lesser payoff. In this paper we experiment with agents who select actions based on expected utility calculations that incorporate the observed frequencies of the actions of the opponent(s). We augment these stochastically greedy agents with an interesting action revelation strategy that involves strategic declaration of one's commitment to an action to avoid worst-case, pessimistic moves. We argue that in certain situations, such apparently risky action revelation can indeed produce better payoffs than a non-revealing approach. In particular, it is possible to obtain Pareto-optimal Nash Equilibrium outcomes. We improve on the outcome efficiency of a previous algorithm and present results over the set of structurally distinct two-person two-action conflict games where the players' preferences form a total order over the possible outcomes. We also present results on a large number of randomly generated payoff matrices of varying sizes and compare the payoffs of strategically revealing learners to payoffs at Nash equilibrium.
1384565
Parallel Sorting in Two-Dimensional VLSI Models of Computation
The gradual refinement of a general approach to two-dimensional sorting, the shear-sort algorithm, to more sophisticated and specialized sorting algorithms on mesh-connected computers is described. The analysis of the shear-sort algorithm gives rise to a novel perspective of two-dimensional sorting, which seems to be a very powerful tool for developing efficient algorithms. The same methods can be extended for sorting in higher dimensions, for example, in the three-dimensional mesh. The concept of clean and dirty rows can be modified to clean and dirty planes (or hyperplanes for dimensions greater than three). Although only two schemes (purely recursive and iterative) are explicitly described, the reader may construct his own algorithm using similar technique and slight modifications. Designing an O(n) algorithm for sorting on a mesh becomes much simpler using the techniques developed. >
207161230
FeedMe: a collaborative alert filtering system
As the number of alerts generated by collaborative applications grows, users receive more unwanted alerts. FeedMe is a general alert management system based on XML feed protocols such as RSS and ATOM. In addition to traditional rule-based alert filtering, FeedMe uses techniques from machine-learning to infer alert preferences based on user feedback. In this paper, we present and evaluate a new collaborative naïve Bayes filtering algorithm. Using FeedMe, we collected alert ratings from 33 users over 29 days. We used the data to design and verify the accuracy of the filtering algorithm and provide insights into alert prediction.
16079000
Searching for optimal coalition structures
Coalition formation has been a very active area of research in multiagent systems. Most of this research has concentrated on decentralized procedures that allow self-interested agents to negotiate the formation of coalitions and division of coalition payoffs. A different line of research has addressed the problem of finding the optimal division of agents into coalitions such that the sum total of the the payoffs to all the coalitions is maximized (Larson and Sandholm, 1999). This is the optimal coalition structure identification problem. Deterministic search algorithms have been proposed and evaluated under the assumption that the performance of a coalition is independent of other coalitions. We use an order-based genetic algorithm (OBGA) as a stochastic search process to identify the optimal coalition structure. We compare the performance of the OBGA with a representative deterministic algorithm presented in the literature. Though the OBGA has no performance guarantees, it is found to dominate the deterministic algorithm in a significant number of problem settings. An additional advantage of the OBGA is its scalability to larger problem sizes and to problems where performance of a coalition depends on other coalitions in the environment.
31766769
Improved selection in totally monotone arrays
This paper's main result is an -time algorithm for computing the kth smallest entry in each row of an m×n totally monotone array. (A two-dimensional array A={a[i, j]} is totally monotone if for all i1<i2 and j1<j2, a[i1, j1]<a[i1, j2] implies a[i2, j1]<a[i2, j2].) For large values of k (in particular, for k=⌈n/2⌉), this algorithm is significantly faster than the O(k(m + n))-time algorithm for the same problem due to Kravets and Park. An immediate consequence of this result is an O(n3/2lg1/2 n)-time algorithm for computing the kth nearest neighbor of each vertex of a convex n-gon. In addition to the main result, we also give an O(n lg m)-time algorithm for computing an approximate median in each row of an m×n totally monotone array; this approximate median is an entry whose rank in its row lies between ⌊n/4⌋ and ⌈3n/4⌉ − 1.
5317468
Strongly Typed Genetic Programming in Evolving Cooperation Strategies
A key concern in genetic programming GP is the size of the state space which must be searched for large and complex problem do mains One method to reduce the state space size is by using Strongly Typed Genetic Programming STGP We applied both GP and STGP to construct cooperation strate gies to be used by multiple predator agents to pursue and capture a prey agent on a grid world This domain has been extensively studied in Distributed Arti cial Intelligence DAI as an easy to describe but di cult to solve cooperation problem The evolved programs from our systems are competitive with manually derived greedy algorithms In particular the STGP paradigm evolved strategies in which the predators were able to achieve their goal without explicitly sens ing the location of other predators or com municating with other predators This is an improvement over previous research in this area The results of our experiments indicate that STGP is able to evolve programs that perform signi cantly better than GP evolved programs In addition the programs gener ated by STGP were easier to understand