A Meta Reinforcement Learning Approach for Predictive Autoscaling in the Cloud
Predictive autoscaling (autoscaling with workload forecasting) is an
important mechanism that supports autonomous adjustment of computing resources
in accordance with fluctuating workload demands in the Cloud. In recent works,
Reinforcement Learning (RL) has been introduced as a promising approach to
learn the resource management policies to guide the scaling actions under the
dynamic and uncertain cloud environment. However, RL methods face the following
challenges in steering predictive autoscaling, such as lack of accuracy in
decision-making, inefficient sampling and significant variability in workload
patterns that may cause policies to fail at test time. To this end, we propose
an end-to-end predictive meta model-based RL algorithm, aiming to optimally
allocate resource to maintain a stable CPU utilization level, which
incorporates a specially-designed deep periodic workload prediction model as
the input and embeds the Neural Process to guide the learning of the optimal
scaling actions over numerous application services in the Cloud. Our algorithm
not only ensures the predictability and accuracy of the scaling strategy, but
also enables the scaling decisions to adapt to the changing workloads with high
sample efficiency. Our method has achieved significant performance improvement
compared to the existing algorithms and has been deployed online at Alipay,
supporting the autoscaling of applications for the world-leading payment
platform.