We present a method for solving service allocation problems in which a set of services must be allocated to a set of agents so as to maximize a global utility. The method is completely distributed so it can scale to any number of services without degradation. We first formalize the service allocation problem and then present a simple hill-climbing, a global hill-climbing, and a bidding-protocol algorithm for solving it. We analyze the expected performance of these algorithms as a function of various problem parameters such as the branching factor and the number of agents. Finally, we use the sensor allocation problem, an instance of a service allocation problem, to show the bidding protocol at work. The simulations also show that phase transition on the expected quality of the solution exists as the amount of communication between agents increases.
There is ongoing work in the field of complexity that attempts to study they dynamics of complex adaptive systems @cite_7 . Our approach is based on ideas borrowed from the use of NK landscapes for the analysis of co-evolving systems. As such, we are using some of the results from that field. However, complexity theory is more concerned with explaining the dynamic behavior of existing systems, while we are more concerned with the engineering of multiagent systems for distributed service allocation.
@cite_7
2108020239
Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multiagent reinforcement learning (MARL). A central issue in the field is the formal statement of the multiagent learning goal. Different viewpoints on this issue have led to the proposal of many different goals, among which two focal points can be distinguished: stability of the agents' learning dynamics, and adaptation to the changing behavior of the other agents. The MARL algorithms described in the literature aim---either explicitly or implicitly---at one of these two goals or at a combination of both, in a fully cooperative, fully competitive, or more general setting. A representative selection of these algorithms is discussed in detail in this paper, together with the specific issues that arise in each category. Additionally, the benefits and challenges of MARL are described along with some of the problem domains where the MARL techniques have been applied. Finally, an outlook for the field is provided. 1. Conceptual outline of current evolutionary theory PART I: ADAPTATION ON THE EDGE OF CHAOS 2. The structure of rugged fitness landscapes 3. Biological implications of rugged fitness landscapes 4. The structure of adaptive landscapes underlying protein evolution 5. Self organization and adaptation in complex systems 6. Coevolving complex systems PART II: THE CRYSTALLIZATION OF LIFE 7. The origins of life: a new view 8. The origin of a connected metabolism 9. Autocatalytic polynucleotide systems: hypercycles, spin glasses and coding 10. Random grammars PART III: ORDER AND ONTOGENY 11. The architecture of genetic regulatory circuits and its evolution 12. Differentiation: the dynamical behaviors of genetic regulatory networks 13. Selection for gene expression in cell type 14. Morphology, maps and the spatial ordering of integrated tissues Service-based systems that are dynamically composed at runtime to provide complex, adaptive functionality are currently one of the main development paradigms in software engineering. However, the Quality of Service (QoS) delivered by these systems remains an important concern, and needs to be managed in an equally adaptive and predictable way. To address this need, we introduce a novel, tool-supported framework for the development of adaptive service-based systems called QoSMOS (QoS Management and Optimization of Service-based systems). QoSMOS can be used to develop service-based systems that achieve their QoS requirements through dynamically adapting to changes in the system state, environment, and workload. QoSMOS service-based systems translate high-level QoS requirements specified by their administrators into probabilistic temporal logic formulae, which are then formally and automatically analyzed to identify and enforce optimal system configurations. The QoSMOS self-adaptation mechanism can handle reliability and performance-related QoS requirements, and can be integrated into newly developed solutions or legacy systems. The effectiveness and scalability of the approach are validated using simulations and a set of experiments based on an implementation of an adaptive service-based system for remote medical assistance. Adaptive evolution, to a large extent, is a complex combinatorial optimization process. In this article we take beginning steps towards developing a general theory of adaptive “walks” via fitter variants in such optimization processes. We introduce the basic idea of a space of entities, each a 1-mutant neighbor of many other entities in the space, and the idea of a fitness ascribed to each entity. Adaptive walks proceed from an initial entity, via fitter neighbors, to locally or globally optimal entities that are fitter than their neighbors. We develop a general theory for the number of local optima, lengths of adaptive walks, and the number of alternative local optima accessible from any given initial entity, for the baseline case of an uncorrelated fitness landscape. Most fitness landscapes are correlated, however. Therefore we develop parts of a universal theory of adaptation on correlated landscapes by adaptive processes that have sufficient numbers of mutations per individual to “jump beyond” the correlation lengths in the underlying landscape. In addition, we explore the statistical character of adaptive walks in two independent complex combinatorial optimization problems, that of evolving a specific cell type in model genetic networks, and that of finding good solutions to the traveling salesman problem. Surprisingly, both show similar statistical features, encouraging the hope that a general theory for adaptive walks on correlated and uncorrelated landscapes can be found. In the final section we explore two limits to the efficacy of selection. The first is new, and surprising: for a wide class of systems, as the complexity of the entities under selection increases, the local optima that are attainable fall progressively closer to the mean properties of the underlying space of entities. This may imply that complex biological systems, such as genetic regulatory systems, are “close” to the mean properties of the ensemble of genomic regulatory systems explored by evolution. The second limit shows that with increasing complexity and a fixed mutation rate, selection often becomes unable to pull an adapting population to those local optima to which connected adaptive walks via fitter variants exist. These beginning steps in theory development are applied to maturation of the immune response, and to the problem of radiation and stasis. Despite the limitations of the adaptive landscape metaphor, we believe that further development along the lines begun here will prove useful. Modern, complex software systems (e-commerce, IT, critical infrastructures, etc.) are increasingly required to continue operation in the face of change, to self-adapt to accommodate shifting user priorities, resource variability, changing environments, and component failures. While manual oversight benefits from global problem contexts and flexible policies, human operators are costly and prone to error. Low-level, embedded mechanisms (exceptions, time-outs, etc.) are effective and timely for error recovery, but are local in scope to the point-of-failure, application-specific, and costly to modify when adaptation objectives change. An ideal solution leverages domain expertise, provides an end-to-end system perspective, adapts the system in a timely manner, and can be engineered cost-effectively. Architecture-based self-adaptation closes the “loop of control,” using external mechanisms and the architecture model of the target system to adapt the system. An architecture model exposes important system properties and constraints, provides end-to-end problem contexts, and allows principled and automated adaptations. Existing architecture-based approaches specialize support for particular classes of systems and fixed sets of quality-of-service concerns; they are costly to develop for new systems and to evolve for new qualities. To overcome these limitations, we posit this thesis: We can provide software engineers the ability to add and evolve self-adaptation capabilities cost-effectively, for a wide range of software systems, and for multiple objectives, by defining a self-adaptation framework that factors out common adaptation mechanisms and provides explicit customization points to tailor self-adaptation capabilities for particular classes of systems, for multiple quality-of-service objectives. Our approach, embodied in a system called Rainbow, provides an engineering approach and a framework of mechanisms to monitor a target system and its environment, reflect observations into the system's architecture model, detect opportunities for improvements, select a course of action, and effect changes. The framework provides general and reusable infrastructures with well-defined customization points, a set of abstractions, and an adaptation engineering process, focusing engineers on adaptation concerns to systematically customize Rainbow to particular systems. To automate system self-adaptation, Rainbow provides a language, called Stitch, to represent routine human adaptation knowledge using a core set of adaptation concepts. Abstract Recurrence is a fundamental property of dynamical systems, which can be exploited to characterise the system's behaviour in phase space. A powerful tool for their visualisation and analysis called recurrence plot was introduced in the late 1980's. This report is a comprehensive overview covering recurrence based methods and their applications with an emphasis on recent developments. After a brief outline of the theory of recurrences, the basic idea of the recurrence plot with its variations is presented. This includes the quantification of recurrence plots, like the recurrence quantification analysis, which is highly effective to detect, e. g., transitions in the dynamics of systems from time series. A main point is how to link recurrences to dynamical invariants and unstable periodic orbits. This and further evidence suggest that recurrences contain all relevant information about a system's behaviour. As the respective phase spaces of two systems change due to coupling, recurrence plots allow studying and quantifying their interaction. This fact also provides us with a sensitive tool for the study of synchronisation of complex systems. In the last part of the report several applications of recurrence plots in economy, physiology, neuroscience, earth sciences, astrophysics and engineering are shown. The aim of this work is to provide the readers with the know how for the application of recurrence plot based methods in their own field of research. We therefore detail the analysis of data and indicate possible difficulties and pitfalls. Multi-Level Location Problems are generally considered as complex. To deal with these problems we propose an approach based on Holonic MultiAgent Systems (HMAS). HMAS have already proven to be a convenient way to engineer complex systems. This approach was merged with Artificial Potential Fields (APF) mechanims. The solution is obtained simultaneously with the holarchy. The holarchy is thus used to exploit and control the emergence of the solution. This solution is then evaluated to check its relevance according to global objectives represented thanks to a fitness function. This model was efficiently applied to a multi-level distribution system. The goal of this paper is to show how co-word analysis techniques can be used to study interactions between academic and technological research. It is based upon a systematic content analysis of publications in the polymer science field over a period of 15 years. The results concern a.) the evolution of research in different subject areas and the patterns of their interaction; b.) a description of subject area “life cycles”; c.) an analysis of ”research trajectories” given factors of stability and change in a research network; d.) the need to use both science push and technology pull theories to explain the interaction dynamics of a research field. The co-word techniques developed in this paper should help to build a bridge between research in scientometrics and work underway to better understand the economics of innovation. Successful recovery from a disrupted state to maintain optimal performance is a key feature that a resilient complex engineered system should have. In the engineering design community, the current focus of engineering resilience research is primarily directed toward improving overall system performance in the presence of likelihood failures. Little attention has been given to the study of how the system responds during and or after the occurrence of a failure event. This paper proposes the use of control theory as a strategy to enable resilient behavior in complex engineered systems. Control theory has various benefits in its application to a resilient engineered system, with the main advantage being its ability to regulate and govern system states, even while the failure is taking place. In the context of implementation within a complex engineered system, such a controller should be designed such that, when a disturbance occurs, the controller should simultaneously be able to take timely action to correct the shift in system performance. To date, the fusion of control theory with engineering resilience has not been explored in-depth by the engineering design community. This paper, thus, presents a resilience modeling and analysis approach using fundamental control theory. The resilience of a power distribution system is employed as a case study to demonstrate the effectiveness of the proposed approach. The presented study also expects to aid in the concurrent development of resilience functions in complex engineered systems under uncertainty. From the Publisher: Genetic algorithms are playing an increasingly important role in studies of complex adaptive systems, ranging from adaptive agents in economic theory to the use of machine learning techniques in the design of complex devices such as aircraft turbines and integrated circuits. Adaptation in Natural and Artificial Systems is the book that initiated this field of study, presenting the theoretical foundations and exploring applications. In its most familiar form, adaptation is a biological process, whereby organisms evolve by rearranging genetic material to survive in environments confronting them. In this now classic work, Holland presents a mathematical model that allows for the nonlinearity of such complex interactions. He demonstrates the model's universality by applying it to economics, physiological psychology, game theory, and artificial intelligence and then outlines the way in which this approach modifies the traditional views of mathematical genetics. Initially applying his concepts to simply defined artificial systems with limited numbers of parameters, Holland goes on to explore their use in the study of a wide range of complex, naturally occuring processes, concentrating on systems having multiple factors that interact in nonlinear ways. Along the way he accounts for major effects of coadaptation and coevolution: the emergence of building blocks, or schemata, that are recombined and passed on to succeeding generations to provide, innovations and improvements. John H. Holland is Professor of Psychology and Professor of Electrical Engineering and Computer Science at the University of Michigan. He is also Maxwell Professor at the Santa Fe Institute and isDirector of the University of Michigan Santa Fe Institute Advanced Research Program. From household appliances to applications in robotics, engineered systems involving complex dynamics can only be as effective as the algorithms that control them. While Dynamic Programming (DP) has provided researchers with a way to optimally solve decision and control problems involving complex dynamic systems, its practical value was limited by algorithms that lacked the capacity to scale up to realistic problems. However, in recent years, dramatic developments in Reinforcement Learning (RL), the model-free counterpart of DP, changed our understanding of what is possible. Those developments led to the creation of reliable methods that can be applied even when a mathematical model of the system is unavailable, allowing researchers to solve challenging control problems in engineering, as well as in a variety of other disciplines, including economics, medicine, and artificial intelligence. Reinforcement Learning and Dynamic Programming Using Function Approximators provides a comprehensive and unparalleled exploration of the field of RL and DP. With a focus on continuous-variable problems, this seminal text details essential developments that have substantially altered the field over the past decade. In its pages, pioneering experts provide a concise introduction to classical RL and DP, followed by an extensive presentation of the state-of-the-art and novel methods in RL and DP with approximation. Combining algorithm development with theoretical guarantees, they elaborate on their work with illustrative examples and insightful comparisons. Three individual chapters are dedicated to representative algorithms from each of the major classes of techniques: value iteration, policy iteration, and policy search. The features and performance of these algorithms are highlighted in extensive experimental studies on a range of control applications. The recent development of applications involving complex systems has led to a surge of interest in RL and DP methods and the subsequent need for a quality resource on the subject. For graduate students and others new to the field, this book offers a thorough introduction to both the basics and emerging methods. And for those researchers and practitioners working in the fields of optimal and adaptive control, machine learning, artificial intelligence, and operations research, this resource offers a combination of practical algorithms, theoretical analysis, and comprehensive examples that they will be able to adapt and apply to their own work. Access the authors' website at www.dcsc.tudelft.nl rlbook for additional material, including computer code used in the studies and information concerning new developments. Our researches on adaptive systems are inspired by their ability of building by themselves a representation of their surrounding world. Using cooperation as a local criterion of self-organization, we study in a network of neuro-agents the evolution of the system and in the same way, the emergence of a functioning coherent with the environmental feed-back. In this paper we expose the abilities of the neuro-agents that give them the autonomy required to build an ab nihilo topology. And finally we want to emphasize the dynamic of the network organization that is basically its best property of adaptation to a changing environment. The work presented here concerns our approach to self-organization in a dynamic neural network. We present some experiments of emergent learning based on our knowledge of complex systems through multi-agent systems technology and cellular biology. In fact we are most interested in the mechanisms involved in the construction of a network able to stabilize its structure in a given environment, than in the efficiency of the learning supported by this structure. Obviously those two ontogenetic processes are strongly coupled and appear simultaneously as mutual consequences. Under those conditions, we conceive artificial systems presenting an inherent complexity close to biological systems as they are open and as their structure evolves from nothing (at least an empty shell) up to an organized, stable and functional network of neuro-agents. Our working hypothesis tells minimal entities of such a system can be built and given with sufficient behavioral potentialities in order to organize themselves to satisfy the whole system activity. In this study, we want to show that a coherent and useful system can emerge without any global model for its functioning, solely from activity and organizational rules of its sub-parts; only agents are designed, not the topology of the network they will belong to. Our thought process supposes four distinct steps that description will be formerly extended. Firstly the neuro-agent unit has to be designed, their properties, their abilities (calculus, communication) and their knowledge. Secondly, initial system is constituted of a minimal number of unlinked neuro-agents. Those pre-built neuro-agents are only receptors (inputs) and actuators (outputs) of the network. Thirdly the global system has to be plunged in a dynamic and open environment in which it will have to adapt to; regarding neuromimetic systems, this phase suppose This text offers a systematic, rigorous, and unified presentation of evolutionary game theory, covering the core developments of the theory from its inception in biology in the 1970s through recent advances. Evolutionary game theory, which studies the behavior of large populations of strategically interacting agents, is used by economists to make predictions in settings where traditional assumptions about agents' rationality and knowledge may not be justified. Recently, computer scientists, transportation scientists, engineers, and control theorists have also turned to evolutionary game theory, seeking tools for modeling dynamics in multiagent systems. Population Games and Evolutionary Dynamics provides a point of entry into the field for researchers and students in all of these disciplines. The text first considers population games, which provide a simple, powerful model for studying strategic interactions among large numbers of anonymous agents. It then studies the dynamics of behavior in these games. By introducing a general model of myopic strategy revision by individual agents, the text provides foundations for two distinct approaches to aggregate behavior dynamics: the deterministic approach, based on differential equations, and the stochastic approach, based on Markov processes. Key results on local stability, global convergence, stochastic stability, and nonconvergence are developed in detail. Ten substantial appendixes present the mathematical tools needed to work in evolutionary game theory, offering a practical introduction to the methods of dynamic modeling. Accompanying the text are more than 200 color illustrations of the mathematics and theoretical results; many were created using the Dynamo software suite, which is freely available on the author's Web site. Readers are encouraged to use Dynamo to run quick numerical experiments and to create publishable figures for their own research. Large collaborative software engineering projects are interesting examples for evolving complex systems. The complexity of these systems unfolds both in evolving software structures, as well as in the social dynamics and organization of development teams. Due to the adoption of Open Source practices and the increasing use of online support infrastructures, large-scale data sets covering both the social and technical dimension of collaborative software engineering processes are increasingly becoming available. In the analysis of these data, a growing number of studies employ a network perspective, using methods and abstractions from network science to generate insights about software engineering processes. Featuring a collection of inspiring works in this area, with this topical issue, we intend to give an overview of state-of-the-art research. We hope that this collection of articles will stimulate downstream applications of network-based data mining techniques in empirical software engineering. Testing is undisputedly a fundamental verification principle in the software landscape. Today's products require us to effectively handle and test huge, complex systems and in this context to tackle challenging traits like heterogeneity, distribution and controllability to name just a few. The advent of Service-Oriented Architectures with their inherent technological features like dynamics and heterogeneity exacerbated faced challenges, requiring us to evolve our technology. The traditional view of white or black box testing, for example, does not accommodate the multitude of shades of grey one should be able to exploit effectively for system-wide tests. Today, while there are a multitude of approaches for testing single services, there is still few work on methodological system tests for SOAs. In this paper we propose a corresponding workflow for tackling SOA testing and diagnosis, discuss SOA test case generation in more detail, and report preliminary research in that direction. Robotic systems are multi-dimensional entities, combining both hardware and software, that are heavily dependent on, and influenced by, interactions with the real world. They can be variously categorised as embedded, cyber-physical, real-time, hybrid, adaptive and even autonomous systems, with a typical robotic system being likely to contain all of these aspects. The techniques for developing and verifying each of these system varieties are often quite distinct. This, together with the sheer complexity of robotic systems, leads us to argue that diverse formal techniques must be integrated in order to develop, verify, and provide certification evidence for, robotic systems. Furthermore, we propose the fast evolving field of robotics as an ideal catalyst for the advancement of integrated formal methods research, helping to drive the field in new and exciting directions and shedding light on the development of large-scale, dynamic, complex systems. Software systems are decomposed hierarchically, for example, into modules, packages and files. This hierarchical decomposition has a profound influence on evolvability, maintainability and work assignment. Hierarchical decomposition is thus clearly of central concern for empirical software engineering researchers; but it also poses a quandary. At what level do we study phenomena, such as quality, distribution, collaboration and productivity? At the level of files? packages? or modules? How does the level of study affect the truth, meaning, and relevance of the findings? In other fields it has been found that choosing the wrong level might lead to misleading or fallacious results. Choosing a proper level, for study, is thus vitally important for empirical software engineering research; but this issue hasn't thus far been explicitly investigated. We describe the related idea of ecological inference and ecological fallacy from sociology and epidemiology, and explore its relevance to empirical software engineering; we also present some case studies, using defect and process data from 18 open source projects to illustrate the risks of modeling at an aggregation level in the context of defect prediction, as well as in hypothesis testing. We present half field offense, a novel subtask of RoboCup simulated soccer, and pose it as a problem for reinforcement learning. In this task, an offense team attempts to outplay a defense team in order to shoot goals. Half field offense extends keepaway [11], a simpler subtask of RoboCup soccer in which one team must try to keep possession of the ball within a small rectangular region, and away from the opposing team. Both keepaway and half field offense have to cope with the usual problems of RoboCup soccer, such as a continuous state space, noisy actions, and multiple agents, but the latter is a significantly harder multiagent reinforcement learning problem because of sparse rewards, a larger state space, a richer action set, and the sheer complexity of the policy to be learned. We demonstrate that the algorithm that has been successful for keepaway is inadequate to scale to the more complex half field offense task, and present a new algorithm to address the aforementioned problems in multiagent reinforcement learning. The main feature of our algorithm is the use of inter-agent communication, which allows for more frequent and reliable learning updates. We show empirical results verifying that our algorithm registers significantly higher performance and faster learning than the earlier approach. We also assess the contribution of inter-agent communication by considering several variations of the basic learning method. This work is a step further in the ongoing challenge to learn complete team behavior for the RoboCup simulated soccer task. A cyber-physical system consists of a collection of computing devices communicating with one another and interacting with the physical world via sensors and actuators in a feedback loop. Increasingly, such systems are everywhere, from smart buildings to medical devices to automobiles. This textbook offers a rigorous and comprehensive introduction to the principles of design, specification, modeling, and analysis of cyber-physical systems. The book draws on a diverse set of subdisciplines, including model-based design, concurrency theory, distributed algorithms, formal methods of specification and verification, control theory, real-time systems, and hybrid systems, explaining the core ideas from each that are relevant to system design and analysis.The book explains how formal models provide mathematical abstractions to manage the complexity of a system design. It covers both synchronous and asynchronous models for concurrent computation, continuous-time models for dynamical systems, and hybrid systems for integrating discrete and continuous evolution. The role of correctness requirements in the design of reliable systems is illustrated with a range of specification formalisms and the associated techniques for formal verification. The topics include safety and liveness requirements, temporal logic, model checking, deductive verification, stability analysis of linear systems, and real-time scheduling algorithms. Principles of modeling, specification, and analysis are illustrated by constructing solutions to representative design problems from distributed algorithms, network protocols, control design, and robotics.This book provides the rapidly expanding field of cyber-physical systems with a long-needed foundational text by an established authority. It is suitable for classroom use or as a reference for professionals. Emergence is a concept widely used in the sciences, the arts, and engineering. Some effort has been made to formalize it, but it is used in various contexts with different meanings, and a unified theory of emergence is still distant. The ultimate goal of a theory of emergence should include using emergence to model, design, or predict the behavior of multiagent systems. The author proposes a formal definition of a basic type of emergence using a language-theoretic and grammar systems approach. It is shown which types of phenomena can be modeled in this sense and what the consequences are for other more complex phenomena.