A review on stochastic approach for dynamic power management in wireless sensor networks <s> Stochastic optimal control approach <s> The goal of a dynamic power management policy is to reduce the power consumption of an electronic system by putting system components into different states, each representing a certain performance and power consumption level. The policy determines the type and timing of these transitions based on the system history, workload, and performance constraints. In this paper we propose a new abstract model of a power-managed electronic system. We formulate the problem of system-level power management as a controlled optimization problem based on the theories of continuous-time Markov derision processes and stochastic networks. This problem is solved exactly using linear programming or heuristically using "policy iteration." Our method is compared with existing heuristic methods for different workload statistics. Experimental results show that the power management method based on a Markov decision process outperforms heuristic methods by as much as 44% in terms of power dissipation savings for a given level of system performance. <s> BIB001 </s> A review on stochastic approach for dynamic power management in wireless sensor networks <s> Stochastic optimal control approach <s> The effectiveness of stochastic power management relies on the accurate system and workload model and effective policy optimization. Workload modeling is a machine learning procedure that finds the intrinsic pattern of the incoming tasks based on the observed workload attributes. Markov Decision Process (MDP) based model has been widely adopted for stochastic power management because it delivers provable optimal policy. Given a sequence of observed workload attributes, the hidden Markov model (HMM) of the workload is trained. If the observed workload attributes and states in the workload model do not have one-to-one correspondence, the MDP becomes a Partially Observable Markov Decision Process (POMDP). This paper presents a framework of modeling and optimization for stochastic power management using HMM and POMDP. The proposed technique discovers the HMM of the workload by maximizing the likelihood of the observed attribute sequence. The POMDP optimization is formulated and solved as a quadraticly constrained linear programming (QCLP). Compared with traditional optimization technique, which is based on value iteration, the QCLP based optimization provides superior policy by enabling stochastic control. <s> BIB002 </s> A review on stochastic approach for dynamic power management in wireless sensor networks <s> Stochastic optimal control approach <s> Dynamic power management (DPM) is one of the main system-level low-power techniques for portable devices. This study presents a fine-grain Markov modelling approach that enables accurate analysis of system power and latency characteristics with full consideration of mode switching overheads in both processor and power controller. The new approach also makes it possible to incorporate latency analysis in terms of deadline satisfaction. <s> BIB003 </s> A review on stochastic approach for dynamic power management in wireless sensor networks <s> Stochastic optimal control approach <s> We present ACES, an automated server provisioning system that aims to meet workload demand while minimizing energy consumption in data centers. To perform energy-aware server provisioning, ACES faces three key tradeoffs between cost, performance, and reliability: (1) maximizing energy savings vs. minimizing unmet load demand, (2) managing low power draw vs. high transition latencies for multiple power management schemes, and (3) balancing energy savings vs. reliability costs of server components due to on-off cycles. To address these challenges, ACES (1) predicts demand in the near future to turn on servers gradually before they are needed and avoids turning on unnecessary servers to cope with transient load spikes, (2) formulates an optimization problem that minimizes a linear combination of unmet demand and total energy and reliability costs, and uses the program structure to solve the problem efficiently in practice, and (3) constructs an execution plan based on the optimization decisions to transition servers between different power states and actuates them using system and load management interfaces. Our evaluation on three data center workloads shows that ACES's energy savings are close to the optimal and it delivers power proportionality while balancing the tradeoff between energy savings and reliability costs. <s> BIB004
Stochastic schemes are based on Markov decision policies which a power manager uses to direct the power manageable devices about the state change. Established in the memory-less property of Markov decision policy, power manager considers only present state of the devices for taking next state decisions. Here, the term "State" is the power mode of the device. The main function of stochastic approaches is the development and analysis of a system model to direct the system components for suitable operating mode to achieve the maximum power savings. The wireless sensor nodes observe the presence of the random inter-arrival of events on their input. Therefore, the input pattern follows any one of the exponential distribution, Pareto distribution, uniform distribution, normal distribution, Bernoulli distribution and Poisson distributions for modelling [42] . A paper on controllable Markov decision model provides heuristic and stochastic policies with linear programming optimization BIB001 . This work gives the importance to workload statistics for two, three and four state system model. It establishes that the delay in attending an event decreases with increase in timeout duration but the power consumption increases. The advanced Partially Observable Markov Decision Process (POMDP) enabled the Hidden Markov model and event arrival distribution by causing modifications in the likelihood of the observed input sequence and optimizing it BIB002 . The paper focused on the Hidden Markov model (HMM) modelling of the service requester and the rest of the system (including service provider and service queue) is modelled in DTMDP. The authors compared the achieved results for HMM with other models and found 65.4% higher than earlier. In accumulation and fire (A&F) policy based model, the power manager can be entirely shut down when the service provider (SP) is activated. However, the A&F policy does not involve updating tasks  BIB003 . This reduces average power dissipation. The PM latency is extremely small. A finite state Markov model for the server that minimizes the workload demand during energy minimization at the cost of reliability in hosting clusters is presented BIB004 . Thus, the hierarchical approaches are required to scale down the large systems consisting of a number of servers. It is found through literature review that the power manager (PM) is the component which dissipates negligible power. Therefore, there is a need of policy that can consider power consumption of power manager along with the other component power on the system. Markov model based policies are not proven globally optimum for stationary Markovian workload. The accuracy of Markov models for non-stationary workload increases by incorporating a number of power states in the Markov chain. Different techniques of dynamic power management have their different implementation approach depending on the application requirement. The power consumption depends on the workload pattern and the states of the different components on the sensor node. The non-geometric transition times of the states and their complex cost functions can also be improved adding more states into a Markov chain of power managed system. However, this can increase the complexity of the system. Finally, constrained by the coarsegrain power management policy, one can search for a refined policy for the states inside each sensor node component dynamically.