The power grid is a complex and vital system that necessitates careful reliability management. Managing the grid is a difficult problem with multiple time scales of decision making and stochastic behavior due to renewable energy generations, variable demand and unplanned outages. Solving this problem in the face of uncertainty requires a new methodology with tractable algorithms. In this work, we introduce a new model for hierarchical decision making in complex systems. We apply reinforcement learning (RL) methods to learn a proxy, i.e., a level of abstraction, for real-time power grid reliability. We devise an algorithm that alternates between slow time-scale policy improvement, and fast time-scale value function approximation. We compare our results to prevailing heuristics, and show the strength of our method.
Several works in the literature of power systems, operational research and more recently machine learning offer approaches for solving sequential stochastic problems using dynamic programming. Of which, the majority of these works focus on energy storage @cite_22 @cite_33 @cite_15 @cite_34 , unit commitment @cite_37 @cite_5 @cite_20 , and energy market bidding strategies @cite_41 @cite_40 @cite_14 . To our knowledge, no work has been done to use MDPs for assessing the reliability in power grids.
@cite_37 @cite_14 @cite_22 @cite_33 @cite_41 @cite_40 @cite_5 @cite_15 @cite_34 @cite_20
1981585916 2071361493 2064073681 1587228649 1987415322 2259944928 2619529385 2047081066 2340337630 2006267017
This paper presents a new method based on the cost-benefit analysis for optimal sizing of an energy storage system in a microgrid (MG). The unit commitment problem with spinning reserve for MG is considered in this method. Time series and feed-forward neural network techniques are used for forecasting the wind speed and solar radiations respectively and the forecasting errors are also considered in this paper. Two mathematical models have been built for both the islanded and grid-connected modes of MGs. The main problem is formulated as a mixed linear integer problem (MLIP), which is solved in AMPL (A Modeling Language for Mathematical Programming). The effectiveness of the approach is validated by case studies where the optimal system energy storage ratings for the islanded and grid-connected MGs are determined. Quantitative results show that the optimal size of BESS exists and differs for both the grid-connected and islanded MGs in this paper. Considerable developments in the real-time telemetry of demand-side systems allow independent system operators (ISOs) to use reserves provided by demand response (DR) in ancillary service markets. Currently, many ISOs have designed programs to utilize the reserve provided by DR in electricity markets. This paper presents a stochastic model to schedule reserves provided by DR in the wholesale electricity markets. Demand-side reserve is supplied by demand response providers (DRPs), which have the responsibility of aggregating and managing customer responses. A mixed-integer representation of reserve provided by DRPs and its associated cost function are used in the proposed stochastic model. The proposed stochastic model is formulated as a two-stage stochastic mixed-integer programming (SMIP) problem. The first-stage involves network-constrained unit commitment in the base case and the second-stage investigates security assurance in system scenarios. The proposed model would schedule reserves provided by DRPs and determine commitment states of generating units and their scheduled energy and spinning reserves in the scheduling horizon. The proposed approach is applied to two test systems to illustrate the benefits of implementing demand-side reserve in electricity markets. The authors develop a model and a solution technique for the problem of generating electric power when demands are not certain. They also provide techniques for improving the current methods used in solving the traditional unit commitment problem. The solution strategy can be run in parallel due to the separable nature of the relaxation used. Numerical results indicate significant savings in the cost of operating power generating systems when the stochastic model is used instead of the deterministic model. There is growing interest in the use of grid-level storage to smooth variations in supply that are likely to arise with an increased use of wind and solar energy. Energy arbitrage, the process of buying, storing, and selling electricity to exploit variations in electricity spot prices, is becoming an important way of paying for expensive investments into grid-level storage. Independent system operators such as the New York Independent System Operator NYISO require that battery storage operators place bids into an hour-ahead market although settlements may occur in increments as small as five minutes, which is considered near "real-time". The operator has to place these bids without knowing the energy level in the battery at the beginning of the hour and simultaneously accounting for the value of leftover energy at the end of the hour. The problem is formulated as a dynamic program. We describe and employ a convergent approximate dynamic programming ADP algorithm that exploits monotonicity of the value function to find a revenue-generating bidding policy; using optimal benchmarks, we empirically show the computational benefits of the algorithm. Furthermore, we propose a distribution-free variant of the ADP algorithm that does not require any knowledge of the distribution of the price process and makes no assumptions regarding a specific real-time price model. We demonstrate that a policy trained on historical real-time price data from the NYISO using this distribution-free approach is indeed effective. This paper focuses on a scenario where a group of independently-operated investor-owned storage units seek to offer both energy and reserve in the day-ahead as well as the hour-ahead markets. We are particularly interested in the case when a significant portion of the power generated in the grid is from wind and other intermittent renewable energy sources. In this regard, we formulate a stochastic programming framework to choose optimal energy and reserve bids for the storage units. Our design takes into account the fluctuating nature of the market prices due to the randomness in the renewable power generation availability. We show that the formulated stochastic program can be converted into a convex optimization problem and therefore it can be solved efficiently. Our simulation results show that our design can assure profitability of the private investment on storage units. In particular, our design results in much higher profit compared to a similar but deterministic design that simply uses the expected values of the price parameters. We evaluate and compare two common methods, artificial neural networks (ANN) and support vector regression (SVR), for predicting energy productions from a solar photovoltaic (PV) system in Florida 15 min, 1 h and 24 h ahead of time. A hierarchical approach is proposed based on the machine learning algorithms tested. The production data used in this work corresponds to 15 min averaged power measurements collected from 2014. The accuracy of the model is determined using computing error statistics such as mean bias error (MBE), mean absolute error (MAE), root mean square error (RMSE), relative MBE (rMBE), mean percentage error (MPE) and relative RMSE (rRMSE). This work provides findings on how forecasts from individual inverters will improve the total solar power generation forecast of the PV system. Abstract This paper proposes a stochastic and dynamic mixed-integer linear program (SD-MILP) for optimal coordinated bidding of a risk-averse profit-maximizing hydropower producer. The day-ahead, intra-day, and real-time markets are considered. To model and predict day-ahead, intra-day, and real-time prices, the Holt–Winter (HW) and the Generalized Autoregressive Conditional Heteroscedastic (GARCH) predictors are combined using a proposed Markov switch. The discrete behavior of intra-day and real-time prices is modeled as different Markov states. The proposed Markov-based HW–GARCH model with a standard scenario generation-reduction technique is used to capture the uncertainty in day-ahead, intra-day, and real-time prices. The time-dependent conditional value at risk (T-CVaR) is proposed to model the risk of trading in different considered markets. The convex combination of the expected profit and T-CVaR is used as the objective of SD-MILP. The Markov-based HW–GARCH is modeled in Matlab and the SD-MILP is coded in GAMS. The Markov-based HW–GARCH predictor and the SD-MILP are used to develop the bidding curve of a three-reservoir hydropower producer using the electricity prices from the Nordic power market. To further examine the developed models a seven-reservoir hydropower producer is also studied. For these two cases, the coordinated bidding curves are derived and discussed. As more renewable, yet volatile, forms of energy like solar and wind are being incorporated into the grid, the problem of finding optimal control policies for energy storage is becoming increasingly important. These sequential decision problems are often modeled as stochastic dynamic programs, but when the state space becomes large, traditional (exact) techniques such as backward induction, policy iteration, or value iteration quickly become computationally intractable. Approximate dynamic programming (ADP) thus becomes a natural solution technique for solving these problems to near-optimality using significantly fewer computational resources. In this paper, we compare the performance of the following: various approximation architectures with approximate policy iteration (API), approximate value iteration (AVI) with structured lookup table, and direct policy search on a benchmarked energy storage problem (i.e., the optimal solution is computable). This paper proposes a model to determine the optimal size of an energy storage facility from a strategic investor's perspective. This investor seeks to maximize its profit through making strategic planning, i.e., storage sizing, and strategic operational, i.e., offering and bidding, decisions. We consider the uncertainties associated with rival generators’ offering strategies and future load levels in the proposed model. The strategic investment decisions include the sizes of charging device, discharging device, and energy reservoir. The proposed model is a stochastic bi-level optimization problem; the planning and operation decisions are made in the upper-level, and market clearing is modeled in the lower-level under different operating scenarios. To make the proposed model computationally tractable, an iterative solution technique based on Benders’ decomposition is implemented. This provides a master problem and a set of subproblems for each scenario. Each subproblem is recast as an mathematical programs with equilibrium constraints. Numerical results based on real-life market data from Alberta's electricity market are provided. We investigate performance limits of a multiple access communication system with energy harvesting nodes where the utility function is taken to be the long-term average sum-throughput. We assume a causal structure for energy arrivals and study the problem in the continuous time regime. For this setting, we first characterize a storage dam model that captures the dynamics of a battery with energy harvesting and variable transmission power. Using this model, we next establish an upper bound on the throughput problem as a function of battery capacity. We also formulate a nonlinear optimization problem to determine optimal achievable power policies for transmitters. Applying a calculus of variations technique, we then derive Euler-Lagrange equations as necessary conditions for optimum power policies in terms of a system of coupled partial integro-differential equations. Based on a Gauss-Seidel algorithm, we devise an iterative algorithm to solve these equations. We also propose a fixed-point algorithm for the symmetric multiple access setting in which the statistical descriptions of energy harvesters are identical. To further support our iterative algorithms, along with the analysis, comprehensive numerical results are also obtained.