FCM-Type Co-clustering Transfer Reinforcement Learning for Non-Markov Processes
In applying reinforcement learning to continuous space problems, discretization or redefinition of the learning space can be a promising approach. Several methods and algorithms have been introduced to learning agents to respond to this problem. In our previous study, we introduced an FCCM clustering technique into Q-learning (called QL-FCCM) and its transfer learning in the Markov process. Since we could not respond to complicated environments like a non-Markov process, in this study, we propose a method in which an agent updates his Q-table by changing the trade-off ratio, Q-learning and QL-FCCM, based on the damping ratio. We conducted numerical experiments of the single pendulum standing problem and our model resulted in a smooth learning process.
Cloud federation offers new business models to enforce more flexible energy management strategies. Independent Cloud providers are exclusively bounded to the specific energy supplier powering its Data Centers. The situation radically change if we consider a federation of cooperating Cloud providers. In such a context a proper migration of virtual machines among providers can lead to global energy sustainability and cost-saving strategy. In this paper, we discuss a decision system for Cloud federation brokerage able to combine these two strategies. More specifically, we present Multi-Criteria Decision Making (MCDM) algorithm able to discover the most convenient Cloud providers candidate to join a particular energy-aware federation. In the end, modelling different possible real Cloud providers, we demonstrate how the algorithm can accommodate different Cloud federation scenarios characterized by particular energy-aware parameters.