Effective service composition using multi agent reinforcement learning
As online services may keep evolving, service composition should maintain certain adaptivity especially for a dynamic composition environment. Meanwhile, the large number of potential candidate services poses scalability concerns, which demand efficient composition solutions. This paper presents a multi agent reinforcement learning model for Web service composition that effectively addresses the above challenges. In particular, we model a service composition as a Markov Decision Process. Based on the model, agents in a team would benefit from one another. In contrast to single agent reinforcement learning, our method can speed up the convergence to an optimal policy. We develop two multi agent reinforcement learning algorithms. The first one introduces the concept of articulate state and distributed Q learning to speed up the convergence time. The second one proposes the experience sharing strategy to improve the efficiency of learning. As the learning process continues throughout the life cycle of a service composition, our algorithms can automatically adapt to the change of environment and the evolving component services. We conduct a simulation study to compare our algorithm with other similar reinforcement learning algorithms, including the traditional Q learning algorithm, a multi agent Sarsa algorithm, a Q learning algorithm based on gaussian process, and a multi agent Q learning algorithm, to justify the effectiveness of our model and algorithm.
Knowl. Based Syst.