Unexpected increases in demand and most of all flash crowds are considered the bane of every web application as they may cause intolerable delays or even service unavailability. Proper quality of service policies must guarantee rapid reactivity and responsiveness even in such critical situations. Previous solutions fail to meet common performance requirements when the system has to face sudden and unpredictable surges of traffic. Indeed they often rely on a proper setting of key parameters which requires laborious manual tuning, preventing a fast adaptation of the control policies. We contribute an original Self-* Overload Control (SOC) policy. This allows the system to self-configure a dynamic constraint on the rate of admitted sessions in order to respect service level agreements and maximize the resource utilization at the same time. Our policy does not require any prior information on the incoming traffic or manual configuration of key parameters. We ran extensive simulations under a wide range of operating conditions, showing that SOC rapidly adapts to time varying traffic and self-optimizes the resource utilization. It admits as many new sessions as possible in observance of the agreements, even under intense workload variations. We compared our algorithm to previously proposed approaches highlighting a more stable behavior and a better performance.
The authors of @cite_18 also address a very important decision problem in the design of the monitoring module: the timing of performance control. They propose to adapt the time interval between successive decisions to the size of workload dependent system parameters, such as the processor queue length. The dynamic adjustment of this interval is of primary importance for threshold based policies for which a constant time interval between decisions may lead to an oscillatory behavior in high load scenarios as we show in Section . Simulations reveal that our algorithm is not subject to oscillations and shows a very little dependence on the time interval between decisions.
@cite_18
2153466231
We present a new family of join algorithms, called ripple joins, for online processing of multi-table aggregation queries in a relational database management system (DBMS). Such queries arise naturally in interactive exploratory decision-support applications. Traditional offline join algorithms are designed to minimize the time to completion of the query. In contrast, ripple joins are designed to minimize the time until an acceptably precise estimate of the query result is available, as measured by the length of a confidence interval. Ripple joins are adaptive, adjusting their behavior during processing in accordance with the statistical properties of the data. Ripple joins also permit the user to dynamically trade off the two key performance factors of on-line aggregation: the time between successive updates of the running aggregate, and the amount by which the confidence-interval length decreases at each update. We show how ripple joins can be implemented in an existing DBMS using iterators, and we give an overview of the methods used to compute confidence intervals and to adaptively optimize the ripple join “aspect-ratio” parameters. In experiments with an initial implementation of our algorithms in the POSTGRES DBMS, the time required to produce reasonably precise online estimates was up to two orders of magnitude smaller than the time required for the best offline join algorithms to produce exact answers. Monitoring Wireless Sensor Networks (WSNs) are composed of sensor nodes that report temperature, relative humidity, and other environmental parameters. The time between two successive measurements is a critical parameter to set during the WSN configuration because it can impact the WSN's lifetime, the wireless medium contention and the quality of the reported data. As trends in monitored parameters can significantly vary between scenarios and within time, identifying a sampling interval suitable for several cases is also challenging. In this work, we propose a dynamic sampling rate adaptation scheme based on reinforcement learning, able to tune sensors' sampling interval on-the-fly, according to environmental conditions and application requirements. The primary goal is to set the sampling interval to the best value possible so as to avoid oversampling and save energy, while not missing environmental changes that can be relevant for the application. In simulations, our mechanism could reduce up to 73 the total number of transmissions compared to a fixed strategy and, simultaneously, keep the average quality of information provided by the WSN. The inherent flexibility of the reinforcement learning algorithm facilitates its use in several scenarios, so as to exploit the broad scope of the Internet of Things. We consider optimal load balancing in a distributed computing environment consisting of homogeneous unreliable processors. Each processor receives its own sequence of tasks from outside users, some of which can be redirected to the other processors. Processing times are independent and identically distributed with an arbitrary distribution. The arrival sequence of outside tasks to each processor may be arbitrary as long as it is independent of the state of the system. Processors may fail, with arbitrary failure and repair processes that are also independent of the state of the system. The only information available to a processor is the history of its decisions for routing work to other processors, and the arrival times of its own arrival sequence. We prove the optimality of the round-robin policy, in which each processor sends all the tasks that can be redirected to each of the other processors in turn. We show that, among all policies that balance workload, round robin stochastically minimizes the nth task completion time for all n, and minimizes response times and queue lengths in a separable increasing convex sense for the entire system. We also show that if there is a single centralized controller, round-robin is the optimal policy, and a single controller using round-robin routing is better than the optimal distributed system in which each processor routes its own arrivals. Again "optimal" and "better" are in the sense of stochastically minimizing task completion times, and minimizing response time and queue lengths in the separable increasing convex sense. A natural optimization model that formulates many online resource allocation problems is the online linear programming LP problem in which the constraint matrix is revealed column by column along with the corresponding objective coefficient. In such a model, a decision variable has to be set each time a column is revealed without observing the future inputs, and the goal is to maximize the overall objective function. In this paper, we propose a near-optimal algorithm for this general class of online problems under the assumptions of random order of arrival and some mild conditions on the size of the LP right-hand-side input. Specifically, our learning-based algorithm works by dynamically updating a threshold price vector at geometric time intervals, where the dual prices learned from the revealed columns in the previous period are used to determine the sequential decisions in the current period. Through dynamic learning, the competitiveness of our algorithm improves over the past study of the same problem. We also present a worst case example showing that the performance of our algorithm is near optimal. A natural optimization model that formulates many online resource allocation and revenue management problems is the online linear program (LP) in which the constraint matrix is revealed column by column along with the corresponding objective coefficient. In such a model, a decision variable has to be set each time a column is revealed without observing the future inputs and the goal is to maximize the overall objective function. In this paper, we provide a near-optimal algorithm for this general class of online problems under the assumption of random order of arrival and some mild conditions on the size of the LP right-hand-side input. Specifically, our learning-based algorithm works by dynamically updating a threshold price vector at geometric time intervals, where the dual prices learned from the revealed columns in the previous period are used to determine the sequential decisions in the current period. Due to the feature of dynamic learning, the competitiveness of our algorithm improves over the past study of the same problem. We also present a worst-case example showing that the performance of our algorithm is near-optimal. Abstract The "secretary problem" models a class of optimal stopping decision tasks in which the binary decision to either stop or continue the search only depends on relative ranks. We present this problem in a computer-controlled experiment designed to investigate sequential observation and selection behavior in the context of employer hiring decisions. Our principal objectives are to test the descriptive power of the optimal decision policy; characterize and competitively test simple decision rules, or heuristics, that individuals might use in optimal stopping decision tasks; and examine the sensitivity of the various decision rules by computer simulation. Our simulation results show that the optimal policy is insensitive to moderate deviations from the optimal cutoff value, and that a simple, non-optimal decision rule that counts the number of successive non-candidates performs remarkably well. Model comparisons show that simple cutoff decision rules account for the decisions of the majority of the subjects. The observed tendency to stop the search too early is accounted for by postulating an endogenous cost of time spent in the search. In this paper, we consider a source node that operates over a time varying channel with energy harvesting capability. The goal of the source is to maximize the average number of successfully delivered packets per time slot. The source is able to choose whether to transmit a packet or defer the transmission in each time slot. The decision which is chosen by the source depends on the channel information available and the length of the energy queue. We formulate the problem of finding the optimal policy as a Markovian decision problem. We show some properties of the value function that represents the discounted number of successfully delivered packets per time slot. We prove that the optimal policy is a threshold type policy depending on the state of the channel and the length of the energy queue. We also derive an upper bound for the average number of packets per time slots successfully received by the destination. We show using numerical results that this bound is a tight bound on the performance of the optimal policy. And we consider the case of time varying channel but without channel state information (CSI). Then, we study the impact of channel time varying nature and the availability of CSI. In this case, we show that the optimal policy is a greedy policy. The performance of this greedy policy is also calculated. In this paper, we consider a scenario where an energy harvesting sensor continuously monitors a system and sends time-stamped status updates to a destination. The destination keeps track of the system status through the received updates. We use the metric Age of Information (AoI), the time that has elapsed since the last received update was generated, to measure the “freshness” of the status information available at the destination. Our objective is to design optimal online status update policies to minimize the long-term average AoI, subject to the energy causality constraint at the sensor. We consider three scenarios, i.e., the battery size is infinite, finite, and one unit only, respectively. For the infinite battery scenario, we adopt a best-effort uniform status update policy and show that it minimizes the long-term average AoI. For the finite battery scenario, we adopt an energy-aware adaptive status update policy, and prove that it is asymptotically optimal when the battery size goes to infinity. For the last scenario where the battery size is one, we first show that within a broadly defined class of online policies, the optimal policy should have a renewal structure. We then focus on a renewal interval, and prove that the optimal policy should have a threshold structure, i.e., if the AoI in the system is below a threshold when an energy arrival enters an empty battery, the sensor should store the energy first and then update when the AoI reaches the threshold; otherwise, it updates the status immediately. Simulation results corroborate the theoretical bounds. This paper studies a generalized class of restless multi-armed bandits with hidden states and allow cumulative feedback, as opposed to the conventional instantaneous feedback. We call them lazy restless bandits (LRBs) as the events of decision making are sparser than the events of state transition. Hence, feedback after each decision event is the cumulative effect of the following state transition events. The states of arms are hidden from the decision maker and rewards for actions are state dependent. The decision maker needs to choose one arm in each decision interval, such that the long-term cumulative reward is maximized. As the states are hidden, the decision maker maintains and updates its belief about them. It is shown that LRBs admit an optimal policy which has threshold structure in belief space. The Whittle-index policy for solving the LRB problem is analyzed; indexability of LRBs is shown. Further, the closed-form index expressions are provided for two sets of special cases; for more general cases, an algorithm for index computation is provided. An extensive simulation study is presented; Whittle-index, modified Whittle-index, and myopic policies are compared. The Lagrangian relaxation of the problem provides an upper bound on the optimal value function; it is used to assess the degree of sub-optimality various policies. Energy efficiency is an important issue in wireless networks. We investigate the power management scheme in the IEEE 802.11 based infrastructure WLANs (wireless local area network) in order to find the optimal parameters that can achieve good energy efficiency without degrading other performances. With power management, an AP (Access Point) might temporarily buffer packets whose destination stations are in the Doze state. We focus our study on the behavior of the buffered packets and their impacts on the performance metrics. We model the power management scheme as an M G 1 queue with bulk service and obtain the analytical results for the energy efficiency and the response time performance metrics, which are controlled by the listen interval. Our simulation results are in good agreement with the analysis. Based on the analytical model and simulation, we propose to select the largest listen interval with the satisfaction of the response time requirement. A common approach in coping with multiperiod optimization problems under uncertainty where statistical information is not really enough to support a stochastic programming model, has been to set up and analyze a number of scenarios. The aim then is to identify trends and essential features on which a robust decision policy can be based. This paper develops for the first time a rigorous algorithmic procedure for determining such a policy in response to any weighting of the scenarios. The scenarios are bundled at various levels to reflect the availability of information, and iterative adjustments are made to the decision policy to adapt to this structure and remove the dependence on hindsight. Adaptive traffic signal control, which adjusts traffic signal timing according to real-time traffic, has been shown to be an effective method to reduce traffic congestion. Available works on adaptive traffic signal control make responsive traffic signal control decisions based on human-crafted features (e.g. vehicle queue length). However, human-crafted features are abstractions of raw traffic data (e.g., position and speed of vehicles), which ignore some useful traffic information and lead to suboptimal traffic signal controls. In this paper, we propose a deep reinforcement learning algorithm that automatically extracts all useful features (machine-crafted features) from raw real-time traffic data and learns the optimal policy for adaptive traffic signal control. To improve algorithm stability, we adopt experience replay and target network mechanisms. Simulation results show that our algorithm reduces vehicle delay by up to 47 and 86 when compared to another two popular traffic signal control algorithms, longest queue first algorithm and fixed time control algorithm, respectively. Complex activities typically consist of multiple primitive events happening in parallel or sequentially over a period of time. Understanding such activities requires recognizing not only each individual event but, more importantly, capturing their spatiotemporal dependencies over different time intervals. Most of the current graphical model-based approaches have several limitations. First, time--sliced graphical models such as hidden Markov models (HMMs) and dynamic Bayesian networks are typically based on points of time and they hence can only capture three temporal relations: precedes, follows, and equals. Second, HMMs are probabilistic finite-state machines that grow exponentially as the number of parallel events increases. Third, other approaches such as syntactic and description-based methods, while rich in modeling temporal relationships, do not have the expressive power to capture uncertainties. To address these issues, we introduce the interval temporal Bayesian network (ITBN), a novel graphical model that combines the Bayesian Network with the interval algebra to explicitly model the temporal dependencies over time intervals. Advanced machine learning methods are introduced to learn the ITBN model structure and parameters. Experimental results show that by reasoning with spatiotemporal dependencies, the proposed model leads to a significantly improved performance when modeling and recognizing complex activities involving both parallel and sequential events. A five-fold increase in leakage current is predicted with each technology generation. While Dynamic Voltage Scaling (DVS) is known to reduce dynamic power consumption, it also causes increased leakage energy drain by lengthening the interval over which a computation is carried out. Therefore, for minimization of the total energy, one needs to determine an operating point, called the critical speed. We compute processor slowdown factors based on the critical speed for energy minimization. Procrastination scheduling attempts to maximize the duration of idle intervals by keeping the processor in a sleep shutdown state even if there are pending tasks, within the constraints imposed by performance requirements. Our simulation experiments show that the critical speed slowdown results in up to 5 energy gains over a leakage oblivious dynamic voltage scaling. Procrastination scheduling scheme extends the sleep intervals to up to 5 times, resulting in up to an additional 18 energy gains, while meeting all timing requirements. Verification of temporal logic properties plays a crucial role in proving the desired behaviors of hybrid systems. In this paper, we propose an interval method for verifying the properties described by a bounded linear temporal logic. We relax the problem to allow outputting an inconclusive result when verification process cannot succeed with a prescribed precision, and present an efficient and rigorous monitoring algorithm that demonstrates that the problem is decidable. This algorithm performs a forward simulation of a hybrid automaton, detects a set of time intervals in which the atomic propositions hold, and validates the property by propagating the time intervals. A continuous state at a certain time computed in each step is enclosed by an interval vector that is proven to contain a unique solution. In the experiments, we show that the proposed method provides a useful tool for formal analysis of nonlinear and complex hybrid systems. Time series are difficult to monitor, summarize and predict. Segmentation organizes time series into few intervals having uniform characteristics (flatness, linearity, modality, monotonicity and so on). For scalability, we require fast linear time algorithms. The popular piecewise linear model can determine where the data goes up or down and at what rate. Unfortunately, when the data does not follow a linear model, the computation of the local slope creates overfitting. We propose an adaptive time series model where the polynomial degree of each interval vary (constant, linear and so on). Given a number of regressors, the cost of each interval is its polynomial degree: constant intervals cost 1 regressor, linear intervals cost 2 regressors, and so on. Our goal is to minimize the Euclidean (l_2) error for a given model complexity. Experimentally, we investigate the model where intervals can be either constant or linear. Over synthetic random walks, historical stock market prices, and electrocardiograms, the adaptive model provides a more accurate segmentation than the piecewise linear model without increasing the cross-validation error or the running time, while providing a richer vocabulary to applications. Implementation issues, such as numerical stability and real-world performance, are discussed. Prediction intervals (PIs) have been proposed in the literature to provide more information by quantifying the level of uncertainty associated to the point forecasts. Traditional methods for construction of neural network (NN) based PIs suffer from restrictive assumptions about data distribution and massive computational loads. In this paper, we propose a new, fast, yet reliable method for the construction of PIs for NN predictions. The proposed lower upper bound estimation (LUBE) method constructs an NN with two outputs for estimating the prediction interval bounds. NN training is achieved through the minimization of a proposed PI-based objective function, which covers both interval width and coverage probability. The method does not require any information about the upper and lower bounds of PIs for training the NN. The simulated annealing method is applied for minimization of the cost function and adjustment of NN parameters. The demonstrated results for 10 benchmark regression case studies clearly show the LUBE method to be capable of generating high-quality PIs in a short time. Also, the quantitative comparison with three traditional techniques for prediction interval construction reveals that the LUBE method is simpler, faster, and more reliable. This paper presents a model for microgrid optimal scheduling considering multi-period islanding constraints. The objective of the problem is to minimize the microgrid total operation cost which comprises the generation cost of local resources and cost of energy purchase from the main grid. The microgrid optimal scheduling problem is decomposed into a grid-connected operation master problem and an islanded operation subproblem. The microgrid capability in operating in the islanded mode for multiple hours is scrutinized by a T-τ islanding criterion. The integer scheduling decisions determined in the master problem will be examined against the microgrid islanding feasibility in the subproblem. The scheduling decisions will be revised using proper islanding cuts if sufficient generation is not available to guarantee a feasible islanding. Islanding cuts will revise generating units, energy storage systems, and adjustable loads schedules. Any change in the schedule of adjustable loads outside the operating time interval specified by consumers is penalized by an inconvenience factor in the objective. Numerical simulations demonstrate the effectiveness of the proposed microgrid optimal scheduling model and explore its economic and reliability merits. A novel control strategy for multi-agent coordination with event-based broadcasting is presented. In particular, each agent decides itself when to transmit its current state to its neighbors and the local control laws are based on these sampled state measurements. Three scenarios are analyzed: Networks of single-integrator agents with and without communication delays, and networks of double-integrator agents. The novel event-based scheduling strategy bounds each agent's measurement error by a time-dependent threshold. For each scenario it is shown that the proposed control strategy guarantees either asymptotic convergence to average consensus or convergence to a ball centered at the average consensus. Moreover, it is shown that the inter-event intervals are lower-bounded by a positive constant. Numerical simulations show the effectiveness of the novel event-based control strategy and how it compares to time-scheduled control. We study an admissions control problem, where a queue with service rate @math receives incoming jobs at rate @math , and the decision maker is allowed to redirect away jobs up to a rate of @math , with the objective of minimizing the time-average queue length. We show that the amount of information about the future has a significant impact on system performance, in the heavy-traffic regime. When the future is unknown, the optimal average queue length diverges at rate @math , as @math . In sharp contrast, when all future arrival and service times are revealed beforehand, the optimal average queue length converges to a finite constant, @math , as @math . We further show that the finite limit of @math can be achieved using only a finite lookahead window starting from the current time frame, whose length scales as @math , as @math . This leads to the conjecture of an interesting duality between queuing delay and the amount of information about the future.