This paper develops a unified methodology for probabilistic analysis and optimal control design for jump diffusion processes defined by polynomials. For such systems, the evolution of the moments of the state can be described via a system of linear ordinary differential equations. Typically, however, the moments are not closed and an infinite system of equations is required to compute statistical moments exactly. Existing methods for stochastic analysis, known as closure methods, focus on approximating this infinite system of equations with a finite dimensional system. This work develops an alternative approach in which the higher order terms, which are approximated in closure methods, are viewed as inputs to a finite-dimensional linear control system. Under this interpretation, upper and lower bounds of statistical moments can be computed via convex linear optimal control problems with semidefinite constraints. For analysis of steady-state distributions, this optimal control problem reduces to a static semidefinite program. These same optimization problems extend automatically to stochastic optimal control problems. For minimization problems, the methodology leads to guaranteed lower bounds on the true optimal value. Furthermore, we show how an approximate optimal control strategy can be constructed from the solution of the semidefinite program. The results are illustrated using numerous examples.
Most methods for stochastic optimal control focus on dynamic programming @cite_28 @cite_12 or approximate dynamic programming @cite_15 . More closely related to this work, however, are relaxation methods. Unlike approximate dynamic programming, these methods provide bounds on the achievable optimal values. The most closely related methods focus on occupation measures @cite_9 @cite_32 @cite_16 . These works rely on the general insight that a convex relaxation of an optimization problem can be obtained by examining randomized strategies @cite_20 . Such relaxation methods apply to stochastic optimal control problems, but it is not clear that they can be used for analysis problems. In contrast, our relaxation methodology applies to both cases. A more removed relaxation method is studied in @cite_17 @cite_13 . These works use duality to relax the constraint that controllers depend causally on state information. Also related is the work of @cite_35 @cite_36 which uses a combination of moment dynamics and deterministic optimal control to find approximate solutions to stochastic optimal control problems. This work, however, only considers systems with closed moments. Furthermore, this work assumes a fixed parametric form for the controller, and does not obtain a relaxation of the original stochastic control problem.
@cite_13 @cite_35 @cite_28 @cite_36 @cite_9 @cite_32 @cite_15 @cite_16 @cite_20 @cite_12 @cite_17
2949276956 2127788848 1992595998 2097148013
This paper considers stochastic optimization problems whose objective functions involve powers of random variables. For example, consider the classic Stochastic lp Load Balancing Problem (SLBp): There are @math machines and @math jobs, and known independent random variables @math decribe the load incurred on machine @math if we assign job @math to it. The goal is to assign each jobs to machines in order to minimize the expected @math -norm of the total load on the machines. While convex relaxations represent one of the most powerful algorithmic tools, in problems such as SLBp the main difficulty is to capture the objective function in a way that only depends on each random variable separately. We show how to capture @math -power-type objectives in such separable way by using the @math -function method, introduced by Lata a to relate the moment of sums of random variables to the individual marginals. We show how this quickly leads to a constant-factor approximation for very general subset selection problem with @math -moment objective. Moreover, we give a constant-factor approximation for SLBp, improving on the recent @math -approximation of [, SODA 18]. Here the application of the method is much more involved. In particular, we need to sharply connect the expected @math -norm of a random vector with the @math -moments of its marginals (machine loads), taking into account simultaneously the different scales of the loads that are incurred by an unknown assignment. This paper proposes a novel two-stage optimization method for robust model predictive control (RMPC) with Gaussian disturbance and state estimation error. Since the disturbance is unbounded, it is impossible to achieve zero probability of violating constraints. Our goal is to optimize the expected value of an objective function while limiting the probability of violating any constraints over the planning horizon (joint chance constraint). Prior arts include ellipsoidal relaxation approach [1] and particle control [2], but the former yields very conservative result and the latter is computationally intensive. Our approach divide the optimization problem into two stages; the upper-stage that optimizes risk allocation, and the lower-stage that optimizes control sequence with tightened constraints. The lower-stage is a regular convex optimization, such as linear programming or quadratic programming. The upper-stage is also convex, but the objective function is not always differentiable. We developed a fast descent algorithm for the upper-stage called iterative risk allocation (IRA), which yield much smaller suboptimality than ellipsoidal relaxation method while achieving a substantial speedup compared to and particle control. We develop a mathematical programming approach for the classicalPSPACE-hard restless bandit problem in stochastic optimization. We introduce a hierarchy ofN (whereN is the number of bandits) increasingly stronger linear programming relaxations, the last of which is exact and corresponds to the (exponential size) formulation of the problem as a Markov decision chain, while the other relaxations provide bounds and are efficiently computed. We also propose a priority-index heuristic scheduling policy from the solution to the firstorder relaxation, where the indices are defined in terms of optimal dual variables. In this way we propose a policy and a suboptimality guarantee. We report results of computational experiments that suggest that the proposed heuristic policy is nearly optimal. Moreover, the second-order relaxation is found to provide strong bounds on the optimal value. Robotic systems need to be able to plan control actions that are robust to the inherent uncertainty in the real world. This uncertainty arises due to uncertain state estimation, disturbances, and modeling errors, as well as stochastic mode transitions such as component failures. Chance-constrained control takes into account uncertainty to ensure that the probability of failure, due to collision with obstacles, for example, is below a given threshold. In this paper, we present a novel method for chance-constrained predictive stochastic control of dynamic systems. The method approximates the distribution of the system state using a finite number of particles. By expressing these particles in terms of the control variables, we are able to approximate the original stochastic control problem as a deterministic one; furthermore, the approximation becomes exact as the number of particles tends to infinity. This method applies to arbitrary noise distributions, and for systems with linear or jump Markov linear dynamics, we show that the approximate problem can be solved using efficient mixed-integer linear-programming techniques. We also introduce an important weighting extension that enables the method to deal with low-probability mode transitions such as failures. We demonstrate in simulation that the new method is able to control an aircraft in turbulence and can control a ground vehicle while being robust to brake failures.