Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can significantly downgrade learning performance. Network self-organization and self-management is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864,000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning.
The design of I-EPOS @cite_31 @cite_49 , used in this paper as case study and performance baseline, adopts a hierarchical approach, featuring: (i) bottom-up abstraction -- via the aggregation of plans from the lower level; (ii) partial isolation -- as different tree branches operate in parallel; and (iii) time tuning -- by synchronizing the execution of hierarchical levels. However, the learning process in I-EPOS executes over the entire hierarchy -- sequentially, level-by-level -- rather than being nested within multiple holarchic levels.
@cite_31 @cite_49
2902246376 2725559531
The Internet of Things equips citizens with a phenomenal new means for online participation in sharing economies. When agents self-determine options from which they choose, for instance, their resource consumption and production, while these choices have a collective systemwide impact, optimal decision-making turns into a combinatorial optimization problem known as NP-hard. In such challenging computational problems, centrally managed (deep) learning systems often require personal data with implications on privacy and citizens’ autonomy. This article envisions an alternative unsupervised and decentralized collective learning approach that preserves privacy, autonomy, and participation of multi-agent systems self-organized into a hierarchical tree structure. Remote interactions orchestrate a highly efficient process for decentralized collective learning. This disruptive concept is realized by I-EPOS, the Iterative Economic Planning and Optimized Selections, accompanied by a paradigmatic software artifact. Strikingly, I-EPOS outperforms related algorithms that involve non-local brute-force operations or exchange full information. This article contributes new experimental findings about the influence of network topology and planning on learning efficiency as well as findings on techno-socio-economic tradeoffs and global optimality. Experimental evaluation with real-world data from energy and bike sharing pilots demonstrates the grand potential of collective learning to design ethically and socially responsible participatory sharing economies. The democratization of Internet of Things and ubiquitous computing equips citizens with phenomenal new ways for online participation and decision-making in application domains of smart grids and smart cities. When agents autonomously self-determine the options from which they make choices, while these choices collectively have an overall system-wide impact, an optimal decision-making turns into a combinatorial optimization problem known to be NP-hard. This paper contributes a new generic self-adaptive learning algorithm for a fully decentralized combinatorial optimization: I-EPOS, the Iterative Economic Planning and Optimized Selections. In contrast to related algorithms that simply parallelize computations or big data and deep learning systems that often require personal data and overtake of control with implication on privacy-preservation and autonomy, I-EPOS relies on coordinated local decision-making via structured interactions over tree topologies that involve the exchange of entirely local and aggregated information. Strikingly, the cost-effectiveness of I-EPOS in regards to performance vs. computational and communication cost highly outperforms other related algorithms that involve non-local brute-force operations or exchange of full information. The algorithm is also evaluated using real-world data from two state-of-the-art pilot projects of participatory sharing economies: (i) energy management and (ii) bicycle sharing. The contribution of an I-EPOS open source software suite implemented as a paradigmatic artifact for community aspires to settle a knowledge exchange for the design of new algorithms and application scenarios of sharing economies towards highly participatory and sustainable digital societies.