The power grid is a complex and vital system that necessitates careful reliability management. Managing the grid is a difficult problem with multiple time scales of decision making and stochastic behavior due to renewable energy generations, variable demand and unplanned outages. Solving this problem in the face of uncertainty requires a new methodology with tractable algorithms. In this work, we introduce a new model for hierarchical decision making in complex systems. We apply reinforcement learning (RL) methods to learn a proxy, i.e., a level of abstraction, for real-time power grid reliability. We devise an algorithm that alternates between slow time-scale policy improvement, and fast time-scale value function approximation. We compare our results to prevailing heuristics, and show the strength of our method.
Several works in the literature of power systems, operational research and more recently machine learning offer approaches for solving sequential stochastic problems using dynamic programming. Of which, the majority of these works focus on energy storage @cite_22 @cite_33 @cite_15 @cite_34 , unit commitment @cite_37 @cite_5 @cite_20 , and energy market bidding strategies @cite_41 @cite_40 @cite_14 . To our knowledge, no work has been done to use MDPs for assessing the reliability in power grids.
@cite_37 @cite_14 @cite_22 @cite_33 @cite_41 @cite_40 @cite_5 @cite_15 @cite_34 @cite_20
1981585916 2071361493 2064073681 1587228649
This paper presents a new method based on the cost-benefit analysis for optimal sizing of an energy storage system in a microgrid (MG). The unit commitment problem with spinning reserve for MG is considered in this method. Time series and feed-forward neural network techniques are used for forecasting the wind speed and solar radiations respectively and the forecasting errors are also considered in this paper. Two mathematical models have been built for both the islanded and grid-connected modes of MGs. The main problem is formulated as a mixed linear integer problem (MLIP), which is solved in AMPL (A Modeling Language for Mathematical Programming). The effectiveness of the approach is validated by case studies where the optimal system energy storage ratings for the islanded and grid-connected MGs are determined. Quantitative results show that the optimal size of BESS exists and differs for both the grid-connected and islanded MGs in this paper. Considerable developments in the real-time telemetry of demand-side systems allow independent system operators (ISOs) to use reserves provided by demand response (DR) in ancillary service markets. Currently, many ISOs have designed programs to utilize the reserve provided by DR in electricity markets. This paper presents a stochastic model to schedule reserves provided by DR in the wholesale electricity markets. Demand-side reserve is supplied by demand response providers (DRPs), which have the responsibility of aggregating and managing customer responses. A mixed-integer representation of reserve provided by DRPs and its associated cost function are used in the proposed stochastic model. The proposed stochastic model is formulated as a two-stage stochastic mixed-integer programming (SMIP) problem. The first-stage involves network-constrained unit commitment in the base case and the second-stage investigates security assurance in system scenarios. The proposed model would schedule reserves provided by DRPs and determine commitment states of generating units and their scheduled energy and spinning reserves in the scheduling horizon. The proposed approach is applied to two test systems to illustrate the benefits of implementing demand-side reserve in electricity markets. The authors develop a model and a solution technique for the problem of generating electric power when demands are not certain. They also provide techniques for improving the current methods used in solving the traditional unit commitment problem. The solution strategy can be run in parallel due to the separable nature of the relaxation used. Numerical results indicate significant savings in the cost of operating power generating systems when the stochastic model is used instead of the deterministic model. There is growing interest in the use of grid-level storage to smooth variations in supply that are likely to arise with an increased use of wind and solar energy. Energy arbitrage, the process of buying, storing, and selling electricity to exploit variations in electricity spot prices, is becoming an important way of paying for expensive investments into grid-level storage. Independent system operators such as the New York Independent System Operator NYISO require that battery storage operators place bids into an hour-ahead market although settlements may occur in increments as small as five minutes, which is considered near "real-time". The operator has to place these bids without knowing the energy level in the battery at the beginning of the hour and simultaneously accounting for the value of leftover energy at the end of the hour. The problem is formulated as a dynamic program. We describe and employ a convergent approximate dynamic programming ADP algorithm that exploits monotonicity of the value function to find a revenue-generating bidding policy; using optimal benchmarks, we empirically show the computational benefits of the algorithm. Furthermore, we propose a distribution-free variant of the ADP algorithm that does not require any knowledge of the distribution of the price process and makes no assumptions regarding a specific real-time price model. We demonstrate that a policy trained on historical real-time price data from the NYISO using this distribution-free approach is indeed effective.