Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> The multi-tier implementation has become the industry standard for developing scalable client-server enterprise applications. Since these applications are performance sensitive, effective models for dynamic resource provisioning and for delivering quality of service to these applications become critical. Workloads in such environments are characterized by client sessions of interdependent requests with changing transaction mix and load over time, making model adaptivity to the observed workload changes a critical requirement for model effectiveness. In this work, we apply a regression-based approximation of the CPU demand of client transactions on a given hardware. Then we use this approximation in an analytic model of a simple network of queues, each queue representing a tier, and show the approximation's effectiveness for modeling diverse workloads with a changing transaction mix over time. Using the TPC- W benchmark and its three different transaction mixes we investigate factors that impact the efficiency and accuracy of the proposed performance prediction models. Experimental results show that this regression-based approach provides a simple and powerful solution for efficient capacity planning and resource provisioning of multi-tier applications under changing workload conditions. <s> BIB001 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Massively Multiplayer Online Games (MMOGs) are a class of computationally intensive client-server applications with severe real-time Quality of Service (QoS) requirements, such as the number of updates per second each client needs to receive from the servers for a fluent and realistic experience. To guarantee the QoS requirements, game providers currently over-provision a large amount of their resources, which makes the overall efficiency of provisioning and utilization of resources rather low and prohibits any but the largest providers from joining the market. To address this deficiency, we propose a new prediction-based method for dynamic resource provisioning and scaling of MMOGs in distributed Grid environments. Firstly, a load prediction service anticipates the future game world entity distribution from historical trace data using a fast and flexible neural network-based method. On top of it, we developed generic analytical game load models used to foresee future hot-spots that congest the game servers and make the overall environment fragmented and unplayable. Finally, a resource allocation service performs dynamic load distribution, balancing, and migration of entities that keep the game servers reasonably loaded such that the real-time QoS requirements are maintained. Experimental results based on a realistic simulation environment demonstrate the advantages of our prediction service compared to other conventional methods, especially due to its ability to adapt to different user load patterns, and a reduction of the average over-allocation from 250% (in the case of static over-provisioning) to around 25% using our dynamic provisioning method. <s> BIB002 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Online Internet applications see dynamic workloads that fluctuate over multiple time scales. This paper argues that the non-stationarity in Internet application workloads, which causes the request mix to change over time, can have a significant impact on the overall processing demands imposed on data center servers. We propose a novel mix-aware dynamic provisioning technique that handles both the non-stationarity in the workload as well as changes in request volumes when allocating server capacity in Internet data centers. Our technique employs the k-means clustering algorithm to automatically determine the workload mix and a queuing model to predict the server capacity for a given workload mix. We implement a prototype provisioning system that incorporates our technique and experimentally evaluate its efficacy on a laboratory Linux data center running the TPC-W web benchmark. Our results show that our k-means clustering technique accurately captures workload mix changes in Internet applications. We also demonstrate that mix-aware dynamic provisioning eliminates SLA violations due to under-provisioning with non-stationary web workloads, and that it offers a better resource usage by reducing over-provisioning when compared to a baseline provisioning approach that only reacts to workload volume changes. We also present a case study of our provisioning approach on Amazon's EC2 cloud platform. <s> BIB003 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Dynamic resource provisioning aims at maintaining the end-to-end response time of a web application within a pre-defined SLA. Although the topic has been well studied for monolithic applications, provisioning resources for applications composed of multiple services remains a challenge. When the SLA is violated, one must decide which service(s) should be reprovisioned for optimal effect. We propose to assign an SLA only to the front-end service. Other services are not given any particular response time objectives. Services are autonomously responsible for their own provisioning operations and collaboratively negotiate performance objectives with each other to decide the provisioning service(s). We demonstrate through extensive experiments that our system can add/remove/shift both servers and caches within an entire multi-service application under varying workloads to meet the SLA target and improve resource utilization. <s> BIB004 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Abstract Cloud infrastructures consisting of heterogeneous resources are increasingly being utilized for hosting large-scale distributed applications from diverse users with discrete needs. The multifarious cloud applications impose varied demands for computational resources along with multitude of performance implications. Successful hosting of cloud applications necessitates service providers to take into account the heterogeneity existing in the behavior of users, applications and system resources while respecting the user’s agreed Quality of Service (QoS) criteria. In this work, we propose a QoS-Aware Resource Elasticity (QRE) framework that allows service providers to make an assessment of the application behavior and develop mechanisms that enable dynamic scalability of cloud resources hosting the application components. Experimental results conducted on the Amazon EC2 cloud clearly demonstrate the effectiveness of our approach while complying with the agreed QoS attributes of users. <s> BIB005 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> In this paper, we introduce a decentralized autonomic architecture for multi-tier applications deployed in cloud environments. The architecture maintains the application's service level objective at a predefined level and, implicitly, reduces the cost. The architecture uses a series of autonomic controllers, in which each controller independently regulates a tier of the application. The architecture utilizes feedback loops and implements Proportional, Integrative and Derivative control laws at each autonomic controller. A prototype is described and an initial set of experiments is conducted on a public commercial cloud. The experiments demonstrate the effectiveness of this approach at maintaining a service level objective through the decomposition of an application's aggregate performance into its set of discretely managed component tiers. <s> BIB006 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Elasticity (on-demand scaling) of applications is one of the most important features of cloud computing. This elasticity is the ability to adaptively scale resources up and down in order to meet varying application demands. To date, most existing scaling techniques can maintain applications' Quality of Service (QoS) but do not adequately address issues relating to minimizing the costs of using the service. In this paper, we propose an elastic scaling approach that makes use of cost-aware criteria to detect and analyse the bottlenecks within multi-tier cloud-based applications. We present an adaptive scaling algorithm that reduces the costs incurred by users of cloud infrastructure services, allowing them to scale their applications only at bottleneck tiers, and present the design of an intelligent platform that automates the scaling process. Our approach is generic for a wide class of multi-tier applications, and we demonstrate its effectiveness against other approaches by studying the behaviour of an example e-commerce application using a standard workload benchmark. <s> BIB007 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Replicated services that allow to scale dynamically can adapt to requests load. Choosing the right number of replicas is fundamental to avoid performance worsening when input spikes occur and to save resources when the load is low. Current mechanisms for automatic scaling are mostly based on fixed thresholds on CPU and memory usage, which are not sufficiently accurate and often entail late countermeasures. We propose Make Your Service Elastic (MYSE), an architecture for automatic scaling of generic replicated services based on queuing models for accurate response time estimation. Requests and service times patterns are analyzed to learn and predict over time their distribution so as to allow for early scaling. A novel heuristic is proposed to avoid the flipping phenomenon. We carried out simulations that show promising results for what concerns the effectiveness of our approach. <s> BIB008 </s> Auto-Scaling Web Applications in Clouds: A Taxonomy and Survey <s> High-Level Metrics <s> Applications with a dynamic workload demand need access to a flexible infrastructure to meet performance guarantees and minimize resource costs. While cloud computing provides the elasticity to scale the infrastructure on demand, cloud service providers lack control and visibility of user space applications, making it difficult to accurately scale the underlying infrastructure. Thus, the burden of scaling falls on the user. In this paper, we propose a new cloud service, Dependable Compute Cloud (DC2), that automatically scales the infrastructure to meet the user-specified performance requirements. DC2 employs Kalman filtering to automatically learn the (possibly changing) system parameters for each application, allowing it to proactively scale the infrastructure to meet performance guarantees. DC2 is designed for the cloud it is application-agnostic and does not require any offline application profiling or benchmarking. Our implementation results on OpenStack using a multi-tier application under a range of workload traces demonstrate the robustness and superiority of DC2 over existing rule-based approaches. <s> BIB009
High-level metrics are performance indicators observed at the application layer. Only auto-scalers deployed by service providers can utilize these metrics as they are not visible to cloud providers. Useful metrics to auto-scaling include request rate, average response time, session creation rate, throughput, service time, and request mix. Some metrics, such as request rate, average response time, throughput, and session creation rate, are easy to measure. They alone enable operation of an auto-scaler as they are direct indicators of the performance of the applications and can be used to trigger scaling operations. The easiest method to construct one is to replace utilization metrics in the simple auto-scaler mentioned in the previous section with any of such high-level metrics. However, this approach is not able to accurately estimate the amount of resources needed and often over-or underprovisions resources. Some auto-scalers require obtaining information about request service time and request mix BIB005 BIB003 BIB001 to estimate how much resources are needed. Different from the previous metrics, they cannot directly trigger scaling actions but can be used to support making efficient scaling plans. In addition, these metrics are not straightforward to measure. Service time is the time a server spends on processing the request, which is widely used in the queuing models to approximate the average response time or sojourn time. Except for a few works BIB006 BIB007 that assume this metric as known a priori, to accurately measure it, either offline profiling BIB002 or support from the application BIB008 is required. Therefore, instead of directly probing it, some works use other approaches to approximate it. Kaur et al. BIB005 mentioned the use of past server logs to infer the mean service time. Gandhi et al. BIB009 employed Kalman filters to estimate service time during runtime. Zhang et al. BIB001 used a regression method to make the approximation. Jiang et al.  resorted to profiling each server when it is first online using a small workload without concurrency and then estimating service time through queuing theory. In another work, Jiang et al. BIB004 utilized a feedback control loop to adjust the estimation of service time at runtime. Request mix is hard to measure because an understanding of the application is essential to distinguish different types of requests. Designing a mechanism to accurately classify various types of requests from outside of the application itself is an interesting and challenging problem to be explored.