In this paper, two behavior control architectures for autonomous agents in the form of cross-platform C++ frameworks are presented, the State Controller Library and the Behavior Control Framework. While the former is state-based and generalizes the notion of states and finite state machines to allow for multi-action planning, the latter is behavior-based and exploits a hierarchical structure and the concept of inhibitions to allow for dynamic transitioning. The two frameworks have completely independent implementations, but can be used effectively in tandem to solve behavior control problems on all levels of granularity. Both frameworks have been used to control the NimbRo-OP, a humanoid soccer robot developed by team NimbRo of the University of Bonn.
Maes developed the Agent Network Architecture (ANA) @cite_13 , @cite_21 , also a behavior-based artificial intelligence approach. In this architecture, the agent consists of a distributed and decentralized collection of primitive behaviors, referred to as . The competence modules are divided into groups of incompatible behaviors, and interact based on a network of predecessor, successor and conflictor links in such a way that the various modules are activated and inhibited dynamically when appropriate. Each competence module only implements a single basic primitive behavior however, so the activation network would grow quite large for real-size problems and become slow and overwhelmed by details @cite_13 . Also, it is a difficult task to tune all of the network parameters and activation functions so that the appropriate module(s) for a given situation in general tend to be activated.
@cite_21 @cite_13
2011149019 2395575420 2963000099 2963353797
The goal of my work is to develop and implement an architecture for an autonomous agent, which I refer to as "ANA". An ANA agent consists of a distributed set of "competence modules". Competence modules are linked in a network. A spreading activation process operates on the network to decide what the "relevance" or relative strength of a competence module is in the current context. This process implements a competition among modules for activation energy. The higher the activation energy level of a module, the more likely it is that this module determines what the autonomous agent does or communicates to believe. Learning is a central, completely integrated feature of the architecture. The competence module network is continuously being developed and changed on the basis of experience: links are added and deleted depending on real world observations and new "macro modules" are created whenever a goal is achieved. This paper presents an overview of the architecture. It describes the functionalities that have been implemented, the results that have been obtained with robotic and simulated ANA agents, and finally it discusses (current) limitations and future work. We consider the problem of multiple agents sensing and acting in environments with the goal of maximising their shared utility. In these environments, agents must learn communication protocols in order to share information that is needed to solve the tasks. By embracing deep neural networks, we are able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability. We propose two approaches for learning in these domains: Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL). The former uses deep Q-learning, while the latter exploits the fact that, during learning, agents can backpropagate error derivatives through (noisy) communication channels. Hence, this approach uses centralised learning but decentralised execution. Our experiments introduce new environments for studying the learning of communication protocols and present a set of engineering innovations that are essential for success in these domains. We consider the problem of multiple agents sensing and acting in environments with the goal of maximising their shared utility. In these environments, agents must learn communication protocols in order to share information that is needed to solve the tasks. By embracing deep neural networks, we are able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability. We propose two approaches for learning in these domains: Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL). The former uses deep Q-learning, while the latter exploits the fact that, during learning, agents can backpropagate error derivatives through (noisy) communication channels. Hence, this approach uses centralised learning but decentralised execution. Our experiments introduce new environments for studying the learning of communication protocols and present a set of engineering innovations that are essential for success in these domains. Due to the publicly-known deterministic character- istic of pilot tones, pilot-aware attack, by jamming, nulling and spoofing pilot tones, can significantly paralyze the uplink channel training in large-scale MISO-OFDM systems. To solve this, we in this paper develop an independence-checking coding based (ICCB) uplink training architecture for one-ring scattering scenarios allowing for uniform linear arrays (ULA) deployment. Here, we not only insert randomized pilots on subcarriers for channel impulse response (CIR) estimation, but also diversify and encode subcarrier activation patterns (SAPs) to convey those pilots simultaneously. The coded SAPs, though interfered by arbitrary unknown SAPs in wireless environment, are qualified to be reliably identified and decoded into the original pilots by checking the hidden channel independence existing in sub- carriers. Specifically, an independence-checking coding (ICC) theory is formulated to support the encoding decoding process in this architecture. The optimal ICC code is further devel- oped for guaranteeing a well-imposed estimation of CIR while maximizing the code rate. Based on this code, the identification error probability (IEP) is characterized to evaluate the reliability of this architecture. Interestingly, we discover the principle of IEP reduction by exploiting the array spatial correlation, and prove that zero- IEP, i.e., perfect reliability, can be guaranteed under continuously-distributed mean angle of arrival (AoA). Besides this, a novel closed form of IEP expression is derived in discretely-distributed case. Simulation results finally verify the effectiveness of the proposed architecture.