3526092
Selecting partners
1. I N T R O D U C T I O N In an open environment the goal of a self-interested agent would be to interact with or enter into partnership with those agents that will produce maximal local utility for this agent [5]. We assume that an agent can get one of several payoffs (utilities) for joining a particular coalition which is determined by a static probability distribution over the payoffs for each coalition. We consider choosing a single coalition to interact with repeatedly when the total number of interactions is known ahead of time.
2727968
A bayes net approach to argumentation
Argumentation-based negotiation approaches have been proposed to capture realistic negotiation contexts where all issues being negotiated are not fixed a priori and the negotiatorýs beliefs may be influenced during the negotiation process. We have proposed a novel Bayesian network based argumentation and decision making framework that allows agents to utilize models of other agents. We discuss a decision mechanism for generating the most effective argument to convince the opponent to accept favorable contracts.
10902519
Effects of Local Information on Group Behavior
Researchers in the field of Distributed Artificial Intelligence have studied the effects of local decisionmaking on overall system performance in both cooperative and self-interested agent groups (Bond & Gasser, 1988) . The performance of individual agents depends critically on the quality of information available to it about local and global goals and resources. Whereas in general it is assumed that the more accurate and up-to-date the available information, the better is the expected performance of the individual and the group, this conclusion can be challenged in a number of scenarios. The populace in human societies tend to look for opportunities and search for better opportunities in their environment (Bartos, 1967) . The theory of migration in social behavior and occupational mobility suggests that the stability of the population depends on how an individual chooses its action based on the prevailing circumst antes. As agent designers, we are faced with the problem of developing decision mechanisms that allow agent societies to stabilize in states where system resources are effectively utilized. In this research, we focus on a particular aspect of distributed decision-making: the effect of limited global knowledge on group behavior. The research question that we are asking is the following: Can limited local knowledge be a boon rather than a bane in a multiagent system ? To investigate this issue we use a resource utilization problem where a number of agents are distributed between several identical resources. We assume that the cost of using any resource is directly proportional to its usage. This cost can be due to a delay in processing of the task in hand, or a reduction in the quality of the resource due to congestion. Hence, there is a justified urge in agents to seek out and move to resource with lesser usage. Other researchers have shown that such systems can exhibit oscillatory or chaotic behavior where agents move back and forth between resources (Hogg & Huberman 1991; Kephart, Hogg & Huberman 1989) resulting in ineffective uti1406 AAAI-96 lization of system resources. Whereas asynchrony, heterogeneous agent decision mechanisms, etc. has been suggested as possible means for solving the instability problem, our proposed solution of using locally differing global views is a novel mechanism to introduce asymmetry in group decisions that expedites group stability.
15974834
Learning Pareto-optimal Solutions in 2x2 Conflict Games
Multiagent learning literature has investigated iterated two-player games to develop mechanisms that allow agents to learn to converge on Nash Equilibrium strategy profiles. Such equilibrium configurations imply that no player has the motivation to unilaterally change its strategy. Often, in general sum games, a higher payoff can be obtained by both players if one chooses not to respond myopically to the other player. By developing mutual trust, agents can avoid immediate best responses that will lead to a Nash Equilibrium with lesser payoff. In this paper we experiment with agents who select actions based on expected utility calculations that incorporate the observed frequencies of the actions of the opponent(s). We augment these stochastically greedy agents with an interesting action revelation strategy that involves strategic declaration of one's commitment to an action to avoid worst-case, pessimistic moves. We argue that in certain situations, such apparently risky action revelation can indeed produce better payoffs than a non-revealing approach. In particular, it is possible to obtain Pareto-optimal Nash Equilibrium outcomes. We improve on the outcome efficiency of a previous algorithm and present results over the set of structurally distinct two-person two-action conflict games where the players' preferences form a total order over the possible outcomes. We also present results on a large number of randomly generated payoff matrices of varying sizes and compare the payoffs of strategically revealing learners to payoffs at Nash equilibrium.
13052705
Learning to Coordinate without Sharing Information
Researchers in the field of Distributed Artificial Intelligence (DAI) have been developing efficient mechanisms to coordinate the activities of multiple autonomous agents. The need for coordination arises because agents have to share resources and expertise required to achieve their goals. Previous work in the area includes using sophisticated information exchange protocols, investigating heuristics for negotiation, and developing formal models of possibilities of conflict and cooperation among agent interests. In order to handle the changing requirements of continuous and dynamic environments, we propose learning as a means to provide additional possibilities for effective coordination. We use reinforcement learning techniques on a block pushing problem to show that agents can learn complimentary policies to follow a desired path without any knowledge about each other. We theoretically analyze and experimentally verify the effects of learning rate on system convergence, and demonstrate benefits of using learned coordination knowledge on similar problems. Reinforcement learning based coordination can be achieved in both cooperative and non-cooperative domains, and in domains with noisy communication channels and other stochastic characteristics that present a formidable challenge to using other coordination schemes.
1281859
Unsupervised Surrogate Agents and Search Bias Change in Flexible Distributed Scheduling
Computational infrastructures for cooperative work should contain embedded agents for handling many routine tasks (Galegher, Kraut, Egldo 1990), but as the number of agents increases and the agents become geographically and/or conceptually dispersed, supervision of the agents will become increasingly problematic. We argue that agents should be provided with deep domain knowledge that allows them to make justifiable decisions, rather than shallow models of users to mimic. In this paper, we use the application domain of distributed meeting scheduling to investigate how agents embodying deeper domain knowledge can choose among alternative strategies for searching their calendars in order to create flexible schedules within reasonable cost.
11169442
Linear Time Algorithms for Clustering Problems in Any Dimensions
We generalize the k-means algorithm presented by the authors [14] and show that the resulting algorithm can solve a larger class of clustering problems that satisfy certain properties (existence of a random sampling procedure and tightness). We prove these properties for the k-median and the discrete k-means clustering problems, resulting in O(2(k/e)O(1)dn) time (1+e)-approximation algorithms for these problems. These are the first algorithms for these problems linear in the size of the input (nd for n points in d dimensions), independent of dimensions in the exponent, assuming k and e to be fixed. A key ingredient of the k-median result is a (1+e)-approximation algorithm for the 1-median problem which has running time O(2(1/e)O(1)d). The previous best known algorithm for this problem had linear running time.
158365
Long-distance 802.11b links: performance measurements and experience
The use of 802.11 long-distance links is a cost-effective means of providing wireless connectivity to rural areas. Although deployments in this setting are increasing, a systematic study of the performance of 802.11 in these settings is lacking. The contributions of this paper are two-fold: (a)we present a detailed performance study of a set of long-distance 802.11b links at various layers of the network stack, and (b)we document the various non-obvious experiences during our study.Our study includes eight long-distance links, ranging from 1km to 37km in length. Unlike prior studies of outdoor 802.11 links, we find that the error rate as a function of the received signal strength behaves close to theory. Time correlation of any packet errors is negligible across a range of time-scales. We have observed at least one of the link to be robust to rain and fog. But any interference on the long-distance links can be detrimental to performance. Apart from this however, such long-distance links can be planned to work well with predictable performance. During our measurements, we have observed a few hardware/driver quirks as well as system bottlenecks apart from the wireless link itself. We believe that our measurements and the documentation of our experience will help future network planning as well as protocol design for these networks.
5317468
Strongly Typed Genetic Programming in Evolving Cooperation Strategies
A key concern in genetic programming GP is the size of the state space which must be searched for large and complex problem do mains One method to reduce the state space size is by using Strongly Typed Genetic Programming STGP We applied both GP and STGP to construct cooperation strate gies to be used by multiple predator agents to pursue and capture a prey agent on a grid world This domain has been extensively studied in Distributed Arti cial Intelligence DAI as an easy to describe but di cult to solve cooperation problem The evolved programs from our systems are competitive with manually derived greedy algorithms In particular the STGP paradigm evolved strategies in which the predators were able to achieve their goal without explicitly sens ing the location of other predators or com municating with other predators This is an improvement over previous research in this area The results of our experiments indicate that STGP is able to evolve programs that perform signi cantly better than GP evolved programs In addition the programs gener ated by STGP were easier to understand
16079000
Searching for optimal coalition structures
Coalition formation has been a very active area of research in multiagent systems. Most of this research has concentrated on decentralized procedures that allow self-interested agents to negotiate the formation of coalitions and division of coalition payoffs. A different line of research has addressed the problem of finding the optimal division of agents into coalitions such that the sum total of the the payoffs to all the coalitions is maximized (Larson and Sandholm, 1999). This is the optimal coalition structure identification problem. Deterministic search algorithms have been proposed and evaluated under the assumption that the performance of a coalition is independent of other coalitions. We use an order-based genetic algorithm (OBGA) as a stochastic search process to identify the optimal coalition structure. We compare the performance of the OBGA with a representative deterministic algorithm presented in the literature. Though the OBGA has no performance guarantees, it is found to dominate the deterministic algorithm in a significant number of problem settings. An additional advantage of the OBGA is its scalability to larger problem sizes and to problems where performance of a coalition depends on other coalitions in the environment.
8938271
Kernel-Based Reinforcement Learning
We present a kernel-based approach to reinforcement learning that overcomes the stability problems of temporal-difference learning in continuous state-spaces. First, our algorithm converges to a unique solution of an approximate Bellman's equation regardless of its initialization values. Second, the method is consistent in the sense that the resulting policy converges asymptotically to the optimal policy. Parametric value function estimates such as neural networks do not possess this property. Our kernel-based approach also allows us to show that the limiting distribution of the value function estimate is a Gaussian process. This information is useful in studying the bias-variance tradeoff in reinforcement learning. We find that all reinforcement learning approaches to estimating the value function, parametric or non-parametric, are subject to a bias. This bias is typically larger in reinforcement learning than in a comparable regression problem.