Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can significantly downgrade learning performance. Network self-organization and self-management is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864,000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning.
Another line of relevant related work concerns the initial selection and maintenance of the topology over which the distributed learning process operates. Multi-agent approaches often rely on self-organization by changing their interactions and system structure at runtime. A holonic multi-agent approach for optimizing facility location problems is earlier introduced @cite_50 . Facilities include distribution of bus stops, hospitals or schools within a geographical area. The agents react to mutual attraction and repulsion forces to self-organize into a holarchy. Stable solutions represent optimal facility localization distributions. This self-organization process has a considerable computational and communication cost, in case of remote agents' interactions. In contrast, the holarchic schemes studied in this paper preserve the agents' organizational structure, while the decentralized learning process used for system optimization is self-adapted by localizing the learning span within part of the tree network.
@cite_50
2096145798 2169551554 2408343736 2909254089
In the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate. This paper identifies several challenges responsible for the non-coordination of independent agents: Pareto-selection, non-stationarity, stochasticity, alter-exploration and shadowed equilibria. A selection of multi-agent domains is classified according to those challenges: matrix games, Boutilier's coordination game, predators pursuit domains and a special multi-state game. Moreover, the performance of a range of algorithms for independent reinforcement learners is evaluated empirically. Those algorithms are Q-learning variants: decentralized Q-learning, distributed Q-learning, hysteretic Q-learning, recursive frequency maximum Q-value and win-or-learn fast policy hill climbing. An overview of the learning algorithms' strengths and weaknesses against each challenge concludes the paper and can serve as a basis for choosing the appropriate algorithm for a new domain. Furthermore, the distilled challenges may assist in the design of new learning algorithms that overcome these problems and achieve higher performance in multi-agent applications. A challenging problem in multi-band multi-cell self-organized wireless systems, such as multi-channel Wi-Fi networks, femto pico cells in 3G 4G cellular networks, and more recent wireless networks over TV white spaces, is of distributed resource allocation. This involves four components: channel selection, client association, channel access, and client scheduling. In this paper, we present a unified framework for jointly addressing the four components with the global system objective of maximizing the clients throughput in a proportionally fair manner. Our formulation allows a natural dissociation of the problem into two sub-parts. We show that the first part, involving channel access and client scheduling, is convex and derive a distributed adaptation procedure for achieving Pareto-optimal solution. For the second part, involving channel selection and client association, we develop a Gibbs-sampler based approach for local adaptation to achieve the global objective, as well as derive fast greedy algorithms from it that achieve good solutions. The study of facility location in the presence of self-interested agents has recently emerged as the benchmark problem in the research on mechanism design without money. Here we study the related problem of heterogeneous 2-facility location, that features more realistic assumptions such as: (i) multiple heterogeneous facilities have to be located, (ii) agents' locations are common knowledge and (iii) agents bid for the set of facilities they are interested in. We study the approximation ratio of both deterministic and randomized truthful algorithms when the underlying network is a line. We devise an (n - 1)-approximate deterministic truthful mechanism and prove a constant approximation lower bound. Furthermore, we devise an optimal and truthful (in expectation) randomized algorithm. This paper addresses the challenges of distributed multi-agent optimisation, in environments with potential adversarial agents. In distributed multi-agent optimisation, each agent has a local cost function and the collective goal is to optimise the global cost function, which takes as input, the output of each agent's local cost function. Such optimisation algorithms are used in different fields such as distributed machine learning, distributed robotics, and recently, distributed energy planning. A prominent assumption in most of these algorithms is that all the agents are cooperative, non-faulty, and non-adversarial. Yet, in scenarios with multi-agent systems, such assumptions are not always valid. Recently, there has been some research in the area of resilience and fault-tolerant distributed multi-agent optimisation. However, these studies either assume a fully Byzantine environment, or have their own adversary model with different sets of assumptions about adversary capabilities. This makes the analysis and comparison between the results challenging. This paper presents a review of such algorithms which helps in defining and investigating restricted adversarial behaviour in distributed multi-agent optimisation. In comparison to the Byzantine environments which carry no assumptions about the agent's capability or purpose, in this paper adversaries have restrictions in regard to their knowledge, capabilities, purpose, and impact. This approach makes it easier to provide guarantees about the system performance depending on the specific adversary model and avoid the overly conservative, 'one size fits all' algorithms needed in fully Byzantine environments. Moreover, using the gained insights, previously unexplored aspects of adversarial behaviour in distributed multi-agent optimisation are discovered and indicated as possible future research directions.