The power grid is a complex and vital system that necessitates careful reliability management. Managing the grid is a difficult problem with multiple time scales of decision making and stochastic behavior due to renewable energy generations, variable demand and unplanned outages. Solving this problem in the face of uncertainty requires a new methodology with tractable algorithms. In this work, we introduce a new model for hierarchical decision making in complex systems. We apply reinforcement learning (RL) methods to learn a proxy, i.e., a level of abstraction, for real-time power grid reliability. We devise an algorithm that alternates between slow time-scale policy improvement, and fast time-scale value function approximation. We compare our results to prevailing heuristics, and show the strength of our method.
Several works in the literature of power systems, operational research and more recently machine learning offer approaches for solving sequential stochastic problems using dynamic programming. Of which, the majority of these works focus on energy storage @cite_22 @cite_33 @cite_15 @cite_34 , unit commitment @cite_37 @cite_5 @cite_20 , and energy market bidding strategies @cite_41 @cite_40 @cite_14 . To our knowledge, no work has been done to use MDPs for assessing the reliability in power grids.
@cite_37 @cite_14 @cite_22 @cite_33 @cite_41 @cite_40 @cite_5 @cite_15 @cite_34 @cite_20
1981585916 2071361493 2064073681 1587228649 1987415322 2259944928 2619529385 2047081066 2340337630 2006267017 1993260741 2540490069 2559108640 2114958204 2007853386 2586403932 2104031208 2145334893 2208906386 2039088753
This paper presents a new method based on the cost-benefit analysis for optimal sizing of an energy storage system in a microgrid (MG). The unit commitment problem with spinning reserve for MG is considered in this method. Time series and feed-forward neural network techniques are used for forecasting the wind speed and solar radiations respectively and the forecasting errors are also considered in this paper. Two mathematical models have been built for both the islanded and grid-connected modes of MGs. The main problem is formulated as a mixed linear integer problem (MLIP), which is solved in AMPL (A Modeling Language for Mathematical Programming). The effectiveness of the approach is validated by case studies where the optimal system energy storage ratings for the islanded and grid-connected MGs are determined. Quantitative results show that the optimal size of BESS exists and differs for both the grid-connected and islanded MGs in this paper. Considerable developments in the real-time telemetry of demand-side systems allow independent system operators (ISOs) to use reserves provided by demand response (DR) in ancillary service markets. Currently, many ISOs have designed programs to utilize the reserve provided by DR in electricity markets. This paper presents a stochastic model to schedule reserves provided by DR in the wholesale electricity markets. Demand-side reserve is supplied by demand response providers (DRPs), which have the responsibility of aggregating and managing customer responses. A mixed-integer representation of reserve provided by DRPs and its associated cost function are used in the proposed stochastic model. The proposed stochastic model is formulated as a two-stage stochastic mixed-integer programming (SMIP) problem. The first-stage involves network-constrained unit commitment in the base case and the second-stage investigates security assurance in system scenarios. The proposed model would schedule reserves provided by DRPs and determine commitment states of generating units and their scheduled energy and spinning reserves in the scheduling horizon. The proposed approach is applied to two test systems to illustrate the benefits of implementing demand-side reserve in electricity markets. The authors develop a model and a solution technique for the problem of generating electric power when demands are not certain. They also provide techniques for improving the current methods used in solving the traditional unit commitment problem. The solution strategy can be run in parallel due to the separable nature of the relaxation used. Numerical results indicate significant savings in the cost of operating power generating systems when the stochastic model is used instead of the deterministic model. There is growing interest in the use of grid-level storage to smooth variations in supply that are likely to arise with an increased use of wind and solar energy. Energy arbitrage, the process of buying, storing, and selling electricity to exploit variations in electricity spot prices, is becoming an important way of paying for expensive investments into grid-level storage. Independent system operators such as the New York Independent System Operator NYISO require that battery storage operators place bids into an hour-ahead market although settlements may occur in increments as small as five minutes, which is considered near "real-time". The operator has to place these bids without knowing the energy level in the battery at the beginning of the hour and simultaneously accounting for the value of leftover energy at the end of the hour. The problem is formulated as a dynamic program. We describe and employ a convergent approximate dynamic programming ADP algorithm that exploits monotonicity of the value function to find a revenue-generating bidding policy; using optimal benchmarks, we empirically show the computational benefits of the algorithm. Furthermore, we propose a distribution-free variant of the ADP algorithm that does not require any knowledge of the distribution of the price process and makes no assumptions regarding a specific real-time price model. We demonstrate that a policy trained on historical real-time price data from the NYISO using this distribution-free approach is indeed effective. This paper focuses on a scenario where a group of independently-operated investor-owned storage units seek to offer both energy and reserve in the day-ahead as well as the hour-ahead markets. We are particularly interested in the case when a significant portion of the power generated in the grid is from wind and other intermittent renewable energy sources. In this regard, we formulate a stochastic programming framework to choose optimal energy and reserve bids for the storage units. Our design takes into account the fluctuating nature of the market prices due to the randomness in the renewable power generation availability. We show that the formulated stochastic program can be converted into a convex optimization problem and therefore it can be solved efficiently. Our simulation results show that our design can assure profitability of the private investment on storage units. In particular, our design results in much higher profit compared to a similar but deterministic design that simply uses the expected values of the price parameters. We evaluate and compare two common methods, artificial neural networks (ANN) and support vector regression (SVR), for predicting energy productions from a solar photovoltaic (PV) system in Florida 15 min, 1 h and 24 h ahead of time. A hierarchical approach is proposed based on the machine learning algorithms tested. The production data used in this work corresponds to 15 min averaged power measurements collected from 2014. The accuracy of the model is determined using computing error statistics such as mean bias error (MBE), mean absolute error (MAE), root mean square error (RMSE), relative MBE (rMBE), mean percentage error (MPE) and relative RMSE (rRMSE). This work provides findings on how forecasts from individual inverters will improve the total solar power generation forecast of the PV system. Abstract This paper proposes a stochastic and dynamic mixed-integer linear program (SD-MILP) for optimal coordinated bidding of a risk-averse profit-maximizing hydropower producer. The day-ahead, intra-day, and real-time markets are considered. To model and predict day-ahead, intra-day, and real-time prices, the Holt–Winter (HW) and the Generalized Autoregressive Conditional Heteroscedastic (GARCH) predictors are combined using a proposed Markov switch. The discrete behavior of intra-day and real-time prices is modeled as different Markov states. The proposed Markov-based HW–GARCH model with a standard scenario generation-reduction technique is used to capture the uncertainty in day-ahead, intra-day, and real-time prices. The time-dependent conditional value at risk (T-CVaR) is proposed to model the risk of trading in different considered markets. The convex combination of the expected profit and T-CVaR is used as the objective of SD-MILP. The Markov-based HW–GARCH is modeled in Matlab and the SD-MILP is coded in GAMS. The Markov-based HW–GARCH predictor and the SD-MILP are used to develop the bidding curve of a three-reservoir hydropower producer using the electricity prices from the Nordic power market. To further examine the developed models a seven-reservoir hydropower producer is also studied. For these two cases, the coordinated bidding curves are derived and discussed. As more renewable, yet volatile, forms of energy like solar and wind are being incorporated into the grid, the problem of finding optimal control policies for energy storage is becoming increasingly important. These sequential decision problems are often modeled as stochastic dynamic programs, but when the state space becomes large, traditional (exact) techniques such as backward induction, policy iteration, or value iteration quickly become computationally intractable. Approximate dynamic programming (ADP) thus becomes a natural solution technique for solving these problems to near-optimality using significantly fewer computational resources. In this paper, we compare the performance of the following: various approximation architectures with approximate policy iteration (API), approximate value iteration (AVI) with structured lookup table, and direct policy search on a benchmarked energy storage problem (i.e., the optimal solution is computable). This paper proposes a model to determine the optimal size of an energy storage facility from a strategic investor's perspective. This investor seeks to maximize its profit through making strategic planning, i.e., storage sizing, and strategic operational, i.e., offering and bidding, decisions. We consider the uncertainties associated with rival generators’ offering strategies and future load levels in the proposed model. The strategic investment decisions include the sizes of charging device, discharging device, and energy reservoir. The proposed model is a stochastic bi-level optimization problem; the planning and operation decisions are made in the upper-level, and market clearing is modeled in the lower-level under different operating scenarios. To make the proposed model computationally tractable, an iterative solution technique based on Benders’ decomposition is implemented. This provides a master problem and a set of subproblems for each scenario. Each subproblem is recast as an mathematical programs with equilibrium constraints. Numerical results based on real-life market data from Alberta's electricity market are provided. We investigate performance limits of a multiple access communication system with energy harvesting nodes where the utility function is taken to be the long-term average sum-throughput. We assume a causal structure for energy arrivals and study the problem in the continuous time regime. For this setting, we first characterize a storage dam model that captures the dynamics of a battery with energy harvesting and variable transmission power. Using this model, we next establish an upper bound on the throughput problem as a function of battery capacity. We also formulate a nonlinear optimization problem to determine optimal achievable power policies for transmitters. Applying a calculus of variations technique, we then derive Euler-Lagrange equations as necessary conditions for optimum power policies in terms of a system of coupled partial integro-differential equations. Based on a Gauss-Seidel algorithm, we devise an iterative algorithm to solve these equations. We also propose a fixed-point algorithm for the symmetric multiple access setting in which the statistical descriptions of energy harvesters are identical. To further support our iterative algorithms, along with the analysis, comprehensive numerical results are also obtained. We develop a stochastic dynamic programming model that co-optimizes the use of energy storage for multiple applications, such as energy, capacity, and backup services, while accounting for market and system uncertainty. Using the example of a battery that has been installed in a home as a distributed storage device, we demonstrate the ability of the model to co-optimize services that ‘compete’ for the capacity of the battery. We also show that these multiple uses of a battery can provide substantive value. In this paper, we study resource allocation for a multicarrier-based cognitive radio (CR) network. More specifically, we investigate the problem of secondary users’ energy-efficiency (EE) maximization problem under secondary total power and primary interference constraints. First, assuming cooperation among the secondary base stations (BSs), a centralized approach is considered to solve the EE optimization problem for the CR network where the primary and secondary users are using either orthogonal frequency-division multiplexing (OFDM) or filter bank based multicarrier (FBMC) modulations. We propose an alternating-based approach to solve the joint power-subcarrier allocation problem. More importantly, in the first place, subcarriers are allocated using a heuristic method for a given feasible power allocation. Then, we conservatively approximate the nonconvex power control problem and propose a joint successive convex approximation-Dinkelbach algorithm (SCADA) to efficiently obtain a solution to the nonconvex power control problem. The proposed algorithm is shown to converge to a solution that coincides with the stationary point of the original nonconvex power control subproblem. Moreover, we propose a dual decomposition-based decentralized version of the SCADA. Second, under the assumption of no cooperation among the secondary BSs, we propose a fully distributed power control algorithm from the perspective of game theory. The proposed algorithm is shown to converge to a Nash-equilibrium (NE) point. Moreover, we propose a sufficient condition that guarantees the uniqueness of the achieved NE. Extensive simulation analyses are further provided to highlight the advantages and demonstrate the efficiency of our proposed schemes. This paper addresses the problem of efficiently operating the storage devices in an electricity microgrid featuring photovoltaic (PV) panels with both shortand long-term storage capacities. The problem of optimally activating the storage devices is formulated as a sequential decision making problem under uncertainty where, at every time-step, the uncertainty comes from the lack of knowledge about future electricity consumption and weather dependent PV production. This paper proposes to address this problem using deep reinforcement learning. To this purpose, a specific deep learning architecture has been designed in order to extract knowledge from past consumption and production time series as well as any available forecasts. The approach is empirically illustrated in the case of a residential customer located in Belgium. We present a technique for computing approximately optimal solutions to stochastic resource allocation problems modeled as Markov decision processes (MDPS). We exploit two key properties to avoid explicitly enumerating the very large state and action spaces associated with these problems. Fist, the problems are composed of multiple tasks whose utilities are independent. Second, the actions taken with respect to (or resources allocated to) a task do not influence the status of any other task. We can therefore view each task as an MDP. However these MDPS are weakly coupled by resource constraints: actions selected for one MDP restrict the actions available to others. We describe heuristic techniques for dealing with several classes of constraints that use the solutions for individual MDPS to construct an approximate global solution. We demonstrate this technique on problems involving thousands of tasks, approximating the solution to problems that are far beyond the reach of standard methods. In this paper, a new optimization framework is proposed to coordinate the operation of large, price-maker, and geographically dispersed energy storage battery systems in a nodal transmission-constrained energy market. The energy storage units are assumed to be investor-owned and independently-operated, seeking to maximize their total profit. Various design factors are taken into consideration such as the location, size, efficiency, and charge and discharge rates of the energy storage units as well as the joint impact of the energy storage operations on the locational marginal prices. While the formulated optimization problem is originally nonlinear and hard to solve, nonlinearities are tackled both in the objective function and in the constraints and the problem is transformed into a tractable mixed-integer linear program, for which the global optimal solutions are found for the charge and discharge schedules of each energy storage unit. Both deterministic and stochastic design scenarios are addressed. Various case studies are presented. It is observed that transmission line congestion is often, but not always, desirable for the coordinated storage systems. Locational diversity, practice of arbitrage, robust design, self-scheduling versus economic bidding, and the overall power system performance are also investigated. During the last years, operators of combined heat and power (CHP) plants are challenged by changing circumstances in the electricity markets. While ensuring a stable heat and power supply for customers, CHP plant operators seek to maximise the profitability of the CHP plant operation. We propose a comprehensive concept and illustrate its potential for increasing the profitability when operating a CHP plant with heat storage by participating in multiple electricity markets. For this purpose, the structure of the decision-making process consisting of bid submission, market clearing and CHP plant operation planning is represented. A multistage stochastic mixed-integer linear programming (MILP) model is developed that simultaneously optimises the operation of the CHP plant with heat storage and bidding in sequential electricity markets. Price uncertainty is captured by means of stochastic processes. The proposed concept is applied to the real case of a municipal energy supply company that participates in the German day-ahead spot and balancing market. Results of the case study illustrate how trading in sequential electricity markets can result in an increased profitability of the CHP plant operation, where additional revenues from trading offset higher generation costs. Furthermore, we exemplify how the optimal operation of the CHP plant and heat storage device are adjusted according to revenue potential in multiple electricity markets. In modern world, every industries including the power industry address technological changes. Therefore, it is necessary to keep track of international experiences and activities taking place in every field. This paper provides an overview of the concept of Unit Commitment (UC) problem with a bibliographical survey of relevant background, the present state and potential methodologies used for solving the concern problem. From the literature, UC problem can be categorized based on the system characteristics and policies applied. This paper reviews not only on regulated and deregulated power systems but also on power systems with renewable energy sources and storage system. In addition, it presents a comprehensive review of the methodologies, which covers a wide span of deterministic, meta-heuristic and hybrid approaches including multi-agent system approach. In terms of contribution, it formulates the problem clearly and describes appropriate approaches to solve the problem. The collected literature of methodologies has been divided into many sections, so that new researchers do not face any difficulty in carrying out research on UC problem under both the regulated and deregulated power industries as well as power system with renewable energy sources and storage system for next generation. This paper presents a security-constrained unit commitment (SCUC) algorithm which takes into account the intermittency and volatility of wind power generation. The UC problem is solved in the master problem with the forecasted intermittent wind power generation. Next, possible scenarios are simulated for representing the wind power volatility. The initial dispatch is checked in the subproblem and generation redispatch is considered for satisfying the hourly volatility of wind power in simulated scenarios. If the redispatch fails to mitigate violations, Benders cuts are created and added to the master problem to revise the commitment solution. The iterative process between the commitment problem and the feasibility check subproblem will continue until simulated wind power scenarios can be accommodated by redispatch. Numerical simulations indicate the effectiveness of the proposed SCUC algorithm for managing the security of power system operation by taking into account the intermittency and volatility of wind power generation. Recently, a number of researchers have proposed spectral algorithms for learning models of dynamical systems—for example, Hidden Markov Models (HMMs), Partially Observable Markov Decision Processes (POMDPs), and Transformed Predictive State Representations (TPSRs). These algorithms are attractive since they are statistically consistent and not subject to local optima. However, they are batch methods: they need to store their entire training data set in memory at once and operate on it as a large matrix, and so they cannot scale to extremely large data sets (either many examples or many features per example). In turn, this restriction limits their ability to learn accurate models of complex systems. To overcome these limitations, we propose a new online spectral algorithm, which uses tricks such as incremental Singular Value Decomposition (SVD) and random projections to scale to much larger data sets and more complex systems than previous methods. We demonstrate the new method on an inertial measurement prediction task and a high-bandwidth video mapping task and we illustrate desirable behaviors such as "closing the loop," where the latent state representation changes suddenly as the learner recognizes that it has returned to a previously known place. Many real-time systems, such as battery-operated embedded devices, are energy constrained. A common problem for these systems is how to reduce energy consumption in the system as much as possible while still meeting the deadlines; a commonly used power management mechanism by these systems is dynamic voltage scaling (DVS). Usually, the workloads executed by these systems are variable and, more often than not, unpredictable. Because of the unpredictability of the workloads, one cannot guarantee to minimize the energy consumption in the system. However, if the variability of the workloads can be captured by the probability distribution of the computational requirement of each task in the system, it is possible to achieve the goal of minimizing the expected energy consumption in the system. In this paper, we investigate DVS schemes that aim at minimizing expected energy consumption for frame-based hard real-time systems. Our investigation considers various DVS strategies (i.e., intra-task DVS, inter-task DVS, and hybrid DVS) and both an ideal system model (i.e., assuming unrestricted continuous frequency, well-defined power-frequency relation, and no speed change overhead) and a realistic system model (i.e., the processor provides a set of discrete speeds, no assumption is made on power-frequency relation, and speed change overhead is considered). The highlights of the investigation are two practical DVS schemes: Practical PACE (PPACE) for a single task and Practical Inter-Task DVS (PITDVS2) for general frame-based systems. Evaluation results show that our proposed schemes outperform and achieve significant energy savings over existing schemes.