Energy Management for Cloud Computing: A Survey from Scheduling Perspective of Heuristic, Game Theory and Learning Strategy <s> Learning Based strategy for Energy Efficient Task Scheduling <s> Heterogeneous clusters and grid infrastructures are becoming increasingly popular. In these computing infrastructures, machines have different resources, including memory sizes, disk space, and installed software packages. These differences give rise to a problem of over-provisioning, that is, sub-optimal utilization of a cluster due to users requesting resource capacities greater than what their jobs actually need. Our analysis of a real workload file (LANL CM5) revealed differences of up to two orders of magnitude between requested memory capacity and actual memory usage. This paper presents an algorithm to estimate actual resource capacities used by batch jobs. Such an algorithm reduces the need for users to correctly predict the resources required by their jobs, while at the same time managing the scheduling system to obtain superior utilization of available hardware. The algorithm is based on the Reinforcement Learning paradigm; it learns its estimation policy on-line and dynamically modifies it according to the overall cluster load. The paper includes simulation results which indicate that our algorithm can yield an improvement of over 30% in utilization (overall throughput) of heterogeneous clusters. <s> BIB001 </s> Energy Management for Cloud Computing: A Survey from Scheduling Perspective of Heuristic, Game Theory and Learning Strategy <s> Learning Based strategy for Energy Efficient Task Scheduling <s> Energy consumption in large-scale distributed systems, such as computational grids and clouds gains a lot of attention recently due to its significant performance, environmental and economic implications. These systems consume a massive amount of energy not only for powering them, but also cooling them. More importantly, the explosive increase in energy consumption is not linear to resource utilization as only a marginal percentage of energy is consumed for actual computational works. This energy problem becomes more challenging with uncertainty and variability of workloads and heterogeneous resources in those systems. This paper presents a dynamic scheduling algorithm incorporating reinforcement learning for good performance and energy efficiency. This incorporation helps the scheduler observe and adapt to various processing requirements (tasks) and different processing capacities (resources). The learning process of our scheduling algorithm develops an association between the best action (schedule) and the current state of the environment (parallel system). We have also devised a task-grouping technique to help the decision-making process of our algorithm. The grouping technique is adaptive in nature since it incorporates current workload and energy consumption for the best action. Results from our extensive simulations with varying processing capacities and a diverse set of tasks demonstrate the effectiveness of this learning approach. <s> BIB002 </s> Energy Management for Cloud Computing: A Survey from Scheduling Perspective of Heuristic, Game Theory and Learning Strategy <s> Learning Based strategy for Energy Efficient Task Scheduling <s> This study looks at how increased memory utilisation affects throughput and energy consumption in scientific computing, especially in high-energy physics. Our aim is to minimise energy consumed by a set of jobs without increasing the processing time. The earlier tests indicated that, especially in data analysis, throughput can increase over 100% and energy consumption decrease 50% by processing multiple jobs in parallel per CPU core. Since jobs are heterogeneous, it is not possible to find an optimum value for the number of parallel jobs. A better solution is based on memory utilisation, but finding an optimum memory threshold is not straightforward. Therefore, a fuzzy logic-based algorithm was developed that can dynamically adapt the memory threshold based on the overall load. In this way, it is possible to keep memory consumption stable with different workloads while achieving significantly higher throughput and energy-efficiency than using a traditional fixed number of jobs or fixed memory threshold approaches. <s> BIB003 </s> Energy Management for Cloud Computing: A Survey from Scheduling Perspective of Heuristic, Game Theory and Learning Strategy <s> Learning Based strategy for Energy Efficient Task Scheduling <s> Lately, many academic and industrial fields have shifted research focus from data acquisition to data analysis. This transition has been facilitated by the usage of Machine Learning (ML) techniques to automatically identify patterns and extract non-trivial knowledge from data. The experimental procedures associated with that are usually complex and computationally demanding. Scheduling is a typical method used to decide how to allocate tasks into available resources. An important step for such is to guess how long an application would take to execute. In this paper, we introduce an approach for predicting processing time specifically of ML tasks. It employs a metalearning framework to relate characteristics of datasets and current machine state to actual execution time. An empirical study was conducted using 78 publicly available datasets, 6 ML algorithms and 4 meta-regressors. Experimental results show that our approach outperforms a commonly used baseline method. Statistical tests advise using SVM r as meta-regressor. These achievements indicate the potential of metalearning to tackle the problem and encourage further developments. <s> BIB004 </s> Energy Management for Cloud Computing: A Survey from Scheduling Perspective of Heuristic, Game Theory and Learning Strategy <s> Learning Based strategy for Energy Efficient Task Scheduling <s> In order to reduce energy consumption of data centers while employing infrastructure resource effectively, a comprehensive resource management method using an improved online energy saving mapping algorithm for virtual machines of data centers is proposed. An intelligent feedback management framework is built for online resource optimization. We propose reinforcement learning and threshold based virtual machine migration to find the lowest energy consumption mapping between virtual machines and host of data centers dynamically and avoid the resources contention in physical machines. Our experimental results shows that the proposed effective online energy saving resource optimization methodology can reduce data center energy consumption by at least 30.3% compared to traditional random placement and save at least 10% running time compared to the offline Q-Learning scheduling algorithm. <s> BIB005
There are several types of learning approaches that are applied in task scheduling such as reinforcement-learning (RL), online learning, self-learning and meta-learning and many more BIB002 BIB003 BIB001 BIB004  . The RL scheme is a goal directed learning approach where it enhances its actions through its experience with trial and error BIB002 . Adaptive Reinforcement Learning Based Scheduling (ARLS) by Hussin et al. BIB002 proposed a dynamic scheduling that observes and adapts to various tasks and heterogeneous resources through feedback signal. ARLS provides the scheduling capability by developing associations between the scheduling decisions and the current state of the environment. The scheduling algorithm calculates the feedback signal i.e., reward and penalty in order to assign the task to the right resource. The authors incorporated task grouping technique by categorizing the various priorities tasks into related task group. The ARLS achieved better trade-off between energy consumption and performance in their experiments. The scheduling approach has the potential to be further enhanced with consideration of dependent-task in the scheduling process. Memory-Based Scheduling by Niemi and Hameri BIB003 utilized fuzzy-logic memory based scheduling to dynamically allocate the memory threshold that was based on the overall load in the distributed systems. The fuzzy control is formulated using free memory, with IF x AND y THEN z rule. Then, the result is calculated using a centroid i.e. the canter of mass method to get the appropriate memory threshold. This scheduling adopted the multi-levels scheduling (i.e., cluster level and node level schedulers). The authors also highlighted that computing and communication intensive applications are required to run in parallel in order to improve performance and energy efficiency. Through their experiments, the scheduling strategy improves throughput and reduces energy consumption compared to one job per CPU core scheduling. Online Energy Saving Resource Optimization (OESRO) by Yuan et al. BIB005 proposed comprehensive resource management method for virtual machines (VMs) to improve energy efficiency and performance. Their energy saving mapping algorithm incorporated reinforcement learning into threshold-based VMs migration to dynamically find the lowest energy consumption between VMs and hosts while avoiding resources contention. In the experiments, they used 0.8 as the threshold value, which means, the next VM needs a new suboptimal host if the current host CPU utilization exceeds 80%. It is proven that the scheduling approach achieved at least 30.3% energy efficiency compared to traditional random placement. The author also has adopted the Sarsa algorithm (i.e.; state-action-reward-state-action) which is a kind of online Q-learning algorithm. Sarsa is better in a way that it uses the real-time Q value rather than the highest Q value, to achieve time-precision decisions. The algorithm is resistant to rapid changing workloads (i.e.; number of tasks and requirements). The work, however, does not consider on how the algorithm handle high computational complexity in the reinforcement learning strategy. The learning based scheduling approach is capable to enhance and improve the scheduling decisions from one cycle to another. It is a very powerful feature for task scheduling in heterogeneous distributed systems especially to survive in the long run. However, the learning approach requires a pre-processing time in order to reach the optimal pace in the learning process BIB002 . Task scheduling that adopted this approach has to formulate better analyzing strategy during the learning process, hence it reduces overhead in computation. Table 3 gives the calculation of the energy model used by the learning based scheduling approach discussed above for energy efficiency. The learning process aims to gain many experiences, therefore the scheduler will be able to justify the best scheduling decision. Due to huge integration process between memory and current state in the systems, the learning-based scheduling might leads to high computational complexity. There are a lot of specifications of learning process that still can be explored for energy efficiency. 