Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Virtualization essentially enables multiple operating systems and applications to run on one physical computer by multiplexing hardware resources. A key motivation for applying virtualization is to improve hardware resource utilization while maintaining reasonable quality of service. However, such a goal cannot be achieved without efficient resource management. Though most physical resources, such as processor cores and I/O devices, are shared among virtual machines using time slicing and can be scheduled flexibly based on priority, allocating an appropriate amount of main memory to virtual machines is more challenging. Different applications have different memory requirements. Even a single application shows varied working set sizes during its execution. An optimal memory management strategy under a virtualized environment thus needs to dynamically adjust memory allocation for each virtual machine, which further requires a prediction model that forecasts its host physical memory needs on the fly. This paper introduces MEmory Balancer (MEB) which dynamically monitors the memory usage of each virtual machine, accurately predicts its memory needs, and periodically reallocates host memory. MEB uses two effective memory predictors which, respectively, estimate the amount of memory available for reclaiming without a notable performance drop, and additional memory required for reducing the virtual machine paging penalty. Our experimental results show that our prediction schemes yield high accuracy and low overhead. Furthermore, the overall system throughput can be significantly improved with MEB. <s> BIB001 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Live migration of virtual machines is a key management function in cloud computing. Unfortunately, no live migration progress management system exists in the state-of-theart, leading to (1) guesswork over how long a migration might take and the inability to schedule dependent tasks accordingly; (2) unacceptable application degradation when application components become split over distant cloud datacenters for an arbitrary period during migration; (3) inability to tradeoff application performance and migration time e.g. to finish migration later for less impact on application performance. Pacer is the first migration progress management system that solves these problems. Pacer's techniques are based on robust and lightweight run-time measurements of system and workload characteristics, efficient and accurate analytic models for progress predictions, and online adaptation to maintain user-defined migration objectives for coordinated and timely migrations. Our experiments on a local testbed and on Amazon EC2 show that Pacer is highly effective under a range of application workloads and network conditions. <s> BIB002 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Host-side SSD caches represent a powerful knob for improving and controlling storage performance and improve performance isolation. We present Centaur, as a host-side SSD caching solution that uses cache sizing as a control knob to achieve storage performance goals. Centaur implements dynamically partitioned per-VM caches with per-partition local replacement to provide both lower cache miss rate, better performance isolation and performance control for VM workloads. It uses SSD cache sizing as a universal knob for meeting a variety of workload-specific goals including per-VM latency and IOPS reservations, proportional share fairness, and aggregate optimizations such as minimizing the average latency across VMs. We implemented Centaur for the VMware ESX hyper visor. With Centaur, times for simultaneously booting 28 virtual desktops improve by 42% relative to a non-caching system and by 18% relative to a unified caching system. Centaur also implements per-VM shares for latency with less than 5% error when running micro benchmarks, and enforces latency and IOPS reservations on OLTP workloads with less than 10% error. <s> BIB003 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Virtual machine (VM) scheduling with load balancing in cloud computing aims to allocate VMs to suitable physical machines (PM) and balance the resource usage among all of the PMs. Correct scheduling of cloud hosts is necessary to develop efficient scheduling strategies to appropriately allocate VMs to physical resources. In this regard the use of dynamic forecast of resource usage in each PM can improve the VM scheduling problem. This paper combines ant colony optimization (ACO) and VM dynamic forecast scheduling (VM_DFS), called virtual machine dynamic prediction scheduling via ant colony optimization (VMDPS-ACO), to solve the VM scheduling problem. In this algorithm through analysis of historical memory consumption in each PM, future memory consumption forecast of VMs on that PM and the efficient allocation of VMs on the cloud infrastructure is performed. We experimented the proposed algorithm using Matlab. The performance of the proposed algorithm is compared with VM_DFS [1]. VM_DFS algorithm exploits first fit decreasing (FFD) scheme using corresponding types (i.e. queuing the list of VMs increasingly, decreasingly or randomly) to schedule VMs and assign them to suitable PMs. We experimented the proposed algorithm in both homogeneous and heterogeneous mode. The results indicate, VMDPS-ACO produces lower resource wastage than VM_DFS in both homogenous and heterogeneous modes and better load balancing among PMs. <s> BIB004 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Dynamic Virtual Machine (VM) consolidation is one of the most promising solutions to reduce energy consumption and improve resource utilization in data centers. Since VM consolidation problem is strictly NP-hard, many heuristic algorithms have been proposed to tackle the problem. However, most of the existing works deal only with minimizing the number of hosts based on their current resource utilization and these works do not explore the future resource requirements. Therefore, unnecessary VM migrations are generated and the rate of Service Level Agreement (SLA) violations are increased in data centers. To address this problem, our VM consolidation method which is formulated as a bin-packing problem considers both the current and future utilization of resources. The future utilization of resources is accurately predicted using a k-nearest neighbor regression based model. In this paper, we investigate the effectiveness of VM and host resource utilization predictions in the VM consolidation task using real workload traces. The experimental results show that our approach provides substantial improvement over other heuristic algorithms in reducing energy consumption, number of VM migrations and number of SLA violations. <s> BIB005 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Host-side flash caching has emerged as a promising solution to the scalability problem of virtual machine (VM) storage in cloud computing systems, but it still faces serious limitations in capacity and endurance. This paper presents CloudCache, an on-demand cache management solution to meet VM cache demands and minimize cache wear-out. First, to support on-demand cache allocation, the paper proposes a new cache demand model, Reuse Working Set (RWS), to capture only the data with good temporal locality, and uses the RWS size (RWSS) to model a workload's cache demand. By predicting the RWSS online and admitting only RWS into the cache, CloudCache satisfies the workload's actual cache demand and minimizes the induced wear-out. Second, to handle situations where a cache is insufficient for the VMs' demands, the paper proposes a dynamic cache migration approach to balance cache load across hosts by live migrating cached data along with the VMs. It includes both on-demand migration of dirty data and background migration of RWS to optimize the performance of the migrating VM. It also supports rate limiting on the cache data transfer to limit the impact to the co-hosted VMs. Finally, the paper presents comprehensive experimental evaluations using real-world traces to demonstrate the effectiveness of CloudCache. <s> BIB006 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Solid-state drives (SSDs) have become promising storage components to serve large I/O demands in modern storage systems. Enterprise class (high-end) SSDs are faster and more resilient than client class (low-end) SSDs but they are expensive to be deployed in large scale storage systems. It is an attractive and practical alternative to exploit the high-end SSDs as a cache and low-end SSDs as main storage. <s> BIB007 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Resource allocation in High Performance Computing (HPC) settings is still not easy for end-users due to the wide variety of application and environment configuration options. Users have difficulties to estimate the number of processors and amount of memory required by their jobs, select the queue and partition, and estimate when job output will be available to plan for next experiments. Apart from wasting infrastructure resources by making wrong allocation decisions, overall user response time can also be negatively impacted. Techniques that exploit batch scheduler systems to predict waiting time and runtime of user jobs have already been proposed. However, we observed that such techniques are not suitable for predicting job memory usage. In this paper we introduce a tool to help users predict their memory requirements using machine learning. We describe the integration of the tool with a batch scheduler system, discuss how batch scheduler log data can be exploited to generate memory usage predictions through machine learning, and present results of two production systems containing thousands of jobs. <s> BIB008 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Abstract With the rapid development of cloud computing, how to reduce energy consumption as well as maintain high computation capacity has become a timely and important challenge. Existing Virtual Machines (VMs) scheduling schemes have mainly focused on enhancing the cluster resource utilization and reducing power consumption by improving the legacy “bin-packing” algorithm. However, different resource-intensive applications running on VMs in realistic scenarios have significant effects on the system performance and energy consumption. Furthermore, instantaneous peak loads may lead to a scheduling error, which can significantly impede the energy efficiency of scheduling algorithms. In this paper, we propose a new scheduling approach named P r e A n t P o l i c y that consists of a prediction model based on fractal mathematics and a scheduler on the basis of an improved ant colony algorithm. The prediction model determines whether to trigger the execution of the scheduler by virtue of load trend prediction, and the scheduler is responsible for resource scheduling while minimizing energy consumption under the premise of guaranteeing the Quality-of-Service (QoS). Through extensive analysis and simulation experiments using real workload traces from the compute clusters of Google, the performance results demonstrate that the proposed approach exhibits excellent energy efficiency and resource utilization. Moreover, this approach offers an effective dynamic capacity provisioning model for resource-intensive applications in a heterogeneous computing environment and can reduce the consumption of system resources and energy when scheduling is triggered by instantaneous peak loads. <s> BIB009 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Cloud computing offers simplified system maintenance and scalable resource management with Virtual Machines (VMs). Users access resources of data centers by allocating VMs to hosts. Therefore, to improve the quality of cloud computing environment, not only the conventional multi Quality of Service (QoS) be satisfied, but also specific importance has to be made on certain metrics such as the system accessibility and resource scheduling in a cooperative and dynamic manner. This paper proposes a method called, Moving Average-based Fuzzy Resource Scheduling (MV-FRS) for virtualized cloud environment to optimize the scheduling of resources through virtual machines. Initially, the MV-FRS method starts by predicting the resource (i.e. bandwidth, memory and processing cycle) requirements. Then a measure of relationships between availability of resources and the requirements of resources are made. Finally, a fuzzy control theory is designed to accomplish system accessibility between user cloud requirements and cloud users resources availability. The simulations results demonstrate that the MV-FRS method is able to reduce the total waiting time of cloud user resource requirements and also ensure the feasibility and effectiveness of the overall system accessibility in terms of average success rate and resource usage when running in a cloud computing environment. Moving Average-based Fuzzy Resource Scheduling (MV-FRS) for virtualized cloud environment to optimize the scheduling of resources through virtual machines.A fuzzy control theory is designed for system accessibility between user cloud requirements and cloud users resources.MV-FRS method is to reduce the total waiting time of cloud user resource requirements and effectiveness of the system accessibility. <s> BIB010 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> In recent years, power consumption has become one of the hottest research trends in computer science and industry. Most of the reasons are related to the operational budget and the environmental issues. In this paper, we would like to propose an energy-efficient solution for orchestrating the resource in cloud computing. In nature, the proposed approach firstly predicts the resource utilization of the upcoming period based on the Gaussian process regression method. Subsequently, the convex optimization technique is engaged to compute an appropriate quantity of physical servers for each monitoring window. This quantity of interest is calculated to ensure that a minimum number of servers can still provide an acceptable quality of service. Finally, a corresponding migrating instruction is issued to stack the virtual machines and turn off the idle physical servers to achieve the objective of energy savings. In order to evaluate the proposed method, we conduct the experiments using synthetic data from 29-day period of Google traces and real workload from the Montage open-source toolkit. Through the evaluation, we show that the proposed approach can achieve a significant result in reducing the energy consumption as well as maintaining the system performance. Balance between energy efficiency and quality of service in the cloud computing.Apply prediction technique to enhance the usefulness of monitoring statistics.Design optimal energy efficiency architecture to orchestrate the cloud system.After VM consolidation, adaptively turn-off idle physical machines to save energy. <s> BIB011 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> Storage subsystem is considered as the performance bottleneck of computer systems in data-intensive applications. Solid-State Drives (SSDs) are emerging storage devices which unlike Hard Disk Drives (HDDs), do not have mechanical parts and therefore, have superior performance compared to HDDs. Due to the high cost of SSDs, entirely replacing HDDs with SSDs is not economically justified. Additionally, SSDs can endure a limited number of writes before failing. To mitigate the shortcomings of SSDs while taking advantage of their high performance, SSD caching is practiced in both academia and industry. Previously proposed caching architectures have only focused on either performance or endurance and neglected to address both parameters in suggested architectures. Moreover, the cost, reliability, and power consumption of such architectures is not evaluated. This paper proposes a hybrid I/O caching architecture that while offers higher performance than previous studies, it also improves power consumption with a similar budget. The proposed architecture uses DRAM, Read-Optimized SSD (RO-SSD), and Write-Optimized SSD (WO-SSD) in a three-level cache hierarchy and tries to efficiently redirect read requests to either DRAM or RO-SSD while sending writes to WO-SSD. To provide high reliability, dirty pages are written to at least two devices which removes any single point of failure. The power consumption is also managed by reducing the number of accesses issued to SSDs. The proposed architecture reconfigures itself between performance- and endurance-optimized policies based on the workload characteristics to maintain an effective tradeoff between performance and endurance. We have implemented the proposed architecture on a server equipped with industrial SSDs and HDDs. The experimental results show that as compared to state-of-the-art studies, the proposed architecture improves performance and power consumption by an average of 8 and 28 percent, respectively, and reduces the cost by 5 percent while increasing the endurance cost by 4.7 percent and negligible reliability penalty. <s> BIB012 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. CACHE <s> When deploying memory-intensive applications to public clouds, one important yet challenging problem is selecting a specific instance type whose memory capacity is large enough to prevent out-of-memory errors while the cost is minimized without violating performance requirements. The state-of-the-practice solution is trial and error, causing both performance overhead and additional monetary cost. This paper investigates two memory scaling mechanisms in public cloud: physical memory (good performance and high cost) and virtual memory (degraded performance and no additional cost). In order to analyze the trade-off between performance and cost of the two scaling options, a performance-cost model is developed that is driven by a lightweight analytic prediction approach through a compact representation of the memory footprint. In addition, for those scenarios when the footprint is unavailable, a meta-model based prediction method is proposed using just-in-time migration mechanisms. The proposed techniques have been extensively evaluated with various benchmarks and real-world applications on Amazon Web Services: the performance-cost model is highly accurate with errors ranging from 1% to 4% and the proposed just-in-time migration approach reduces the monetary cost by up to 66%. <s> BIB013
To address the storage bottleneck and to improve VM performance for cloud computing systems, the host-side flash caching was suggested to use. Recent studies have focused on solutions for reducing the capacity and endurance limitations of the flash caching systems. A general scheme of the cache organisation in Cloud is shown in Figure 22 . CloudCache BIB006 is an on-demand cache management solution to meet VM cache demands and minimise cache wear-out. To capture the data with a good temporal locality, it employs a new cache demand model, Reuse Working Set (RWS), which is defined as a set of separate (address) data blocks that the workload reused at least N times in a period of time. In BIB003 host-side SSD caching solution Centaur was presented. This research is mainly focused on the cache partitioning as the key mechanism that eliminates cache wastage. Centaur implements dynamically partitioned per-VM caches with per-partition local replacement to provide lower cache miss rate, better performance isolation and performance control for VM workloads. Partitioning employs online miss rate curves (MRC) construction combined with periodic partitioning and data movement to allocate cache space to individual workloads efficiently. The approach presented in BIB007 consists in sampling the workload and in running them in separate sample caches with different workload classifiers (thresholds) to find the optimal threshold. Based on the estimation of the latency over a lifetime vs threshold curve, the optimal threshold could be selected. A Threelevel I/O Cache Architecture (TICA), which aims to improve the performance and power consumption of SSD-based I/O caching while having minimal impact on the endurance, was presented in BIB012 . TICA employs Read-Optimised SSD (RO-SSD), Write-Optimised SSD (WO-SSD), and DRAM as three levels of cache memory. To balance performance and endurance, a state-machine model that selects one of the existing policies, such as Endurance-Friendly (TICA-EF) for optimising endurance and Write to Endurance Disk (TICA-WED) for improving performance, was implemented. Effective caching schemes should not focus only on estimating the VMs cache size, but also take into account other critical parameters, such as the write policy, the request type, and the workload reuse, which significantly affects the performance and endurance of the SSD cache. Intelligent data prefetching based on access patterns can hide the latencies of the disk access and data transfer by moving the data to DRAM before it is requested. However, the effectiveness of such prefetching strongly depends on the ability to recognise data access patterns and identify corresponding data to be prefetched. VOLUME 7, 2019 H. PREDICTION It is typical for cloud environment meeting unexpected loads, which may lead to overload and performance degradation. The accurate prediction of the future workload enables the cloud providers to provide effective resource management, including memory allocation. In BIB009 a prediction model based on fractal mathematics was used for predicting the usage of CPU and disk I/O bandwidth load. Moreover, there are studies with memory utilisation prediction. In BIB004 an algorithm to predict the memory usage on the PMs to prevent overloading of PMs was used. Autoregression to predict the memory consumption per VM is used when the actual memory consumption in a PM exceeds the preset percentage. The moving average-based resource requirement forecasting algorithm presented in BIB010 is used for the forecast of resource requirement including bandwidth, memory and CPU. The advantage of using this prediction model is to reduce the total waiting time between the forecasting value (resource required) and the actual value (resource availability), thus improving the resource scheduling efficiency. Predictors in BIB001 are used to estimate the amount of memory available for reclaiming without a notable performance drop, and additional memory required for reducing the VM paging penalty. The memory growth predictors were implemented in two ways: the OS statistics based and the LRU-based ones. CloudCache proposed in BIB006 uses classic exponential smoothing and double exponential smoothing methods to predict the cache demand of the next time window. In BIB011 Gaussian process regression (GPR) is employed as a probability framework. How the prediction works depends on the monitoring statistics which reflect the historical utilisation of the cluster facilities. The memory and CPU utilisation prediction results are presented. The prediction tool proposed in BIB008 relies on a combination of machine learning methods, such as Support Vector Machines (SVMs) with Linear and Radial Basis Function (RBF) kernels, Random Forests, Multilayer Perceptrons (MLPs), and k Nearest Neighbors (kNN), to make predictions of job memory usage. A wide range of features collected by the proposed tool is stored in a separate embedded database. The label used in machine learning methods is the maximum memory used discretisation on bins of size 512MiB. Experiments based on job traces from two production environments show that there is no single machine learning method that produces the best predictions. The proposed tool chooses the most promising ones at a given time. The approach presented in BIB013 considers both statistical models and the hint extracted from the application to accurately predict when the memory will error out. Multiple fitting models (e.g. polynomial, exponential) along with applications' own traits extracted from profiling a small portion of execution are considered as inputs. They are also employed to construct a higher-level of the model that is used in migration decision making. Pacer's techniques BIB002 uses analytic models for progress predictions. The prediction of the migration time includes an evaluation of the total number of dirty pages/blocks during migration that is not known before the migration completes. Memory dirty rate prediction is based on the analysis of the snapshot of the dirty bitmap of memory pages. This leads to a large tracking overhead, so instead of continuous monitoring, a sample-based algorithm was proposed. Utilisation prediction-aware best fit decreasing approach described in BIB005 uses a k-nearest neighbour regression-based model for the prediction of the future utilisation of resources, including both CPU and memory. Summarising the aforementioned forecasting methods, it can be noted that the memory prediction is essential in managing cloud resources and can be considered for several objectives. The taxonomy of the memory prediction objectives is presented in Figure 23 . An accurate prediction of VM memory requests is still a challenging problem, especially in a changing environment. Because the hypervisor lacks the knowledge of VM memory access pattern, the virtualisation context makes the prediction even harder. Besides, access patterns can change at runtime, the workflow may include third-party libraries where the access pattern is unknown to the user, and the resources may be shared between multiple application workflows. Therefore, the current trend in the resource management is to use machine learning approaches for more accurate and long-term workload prediction with an attempt to avoid the enormous performance overhead. 