Estimation of model parameters in a dynamic system can be significantly improved with the choice of experimental trajectory. For general nonlinear dynamic systems, finding globally “best” trajectories is typically not feasible; however, given an initial estimate of the model parameters and an initial trajectory, we present a continuous-time optimization method that produces a locally optimal trajectory for parameter estimation in the presence of measurement noise. The optimization algorithm is formulated to find system trajectories that improve a norm on the Fisher information matrix (FIM). A double-pendulum cart apparatus is used to numerically and experimentally validate this technique. In simulation, the optimized trajectory increases the minimum eigenvalue of the FIM by three orders of magnitude, compared with the initial trajectory. Experimental results show that this optimized trajectory translates to an order-of-magnitude improvement in the parameter estimate error in practice.
Discretization of the dynamics has several problems. First and foremost, an arbitrary choice about the discretization needs to be made. Secondly, adaptive time-stepping methods cannot be used, and a discretization appropriate for the initial trajectory cannot be expected to be appropriate for the final trajectory. Lastly, discretization can lead to high-dimensional constrained optimizations (dimensions of @math to @math are common in practical problems) that are impractical to solve numerically. To avoid discretization of continuous dynamics, a class of methods has been developed that relies on sets of basis functions to synthesize optimal controls for the system @cite_2 @cite_9 @cite_34 . These methods allow the full trajectory to be optimized on a continuous-time domain; however, the optimization problem is still subject to a finite set of basis function coefficients. One example of the basis function set includes the Fourier basis, which is used create a class of trajectories over which the optimization can be performed @cite_4 .
@cite_9 @cite_34 @cite_4 @cite_2
2615790994 2343345052 2003095404 2029859487
It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequence-to-sequence models for structured prediction problems to develop policies over discretized spaces. Central to this method is the realization that complex functions over high dimensional spaces can be modeled by neural networks that predict one dimension at a time. Specifically, we show how Q-values and policies over continuous spaces can be modeled using a next step prediction model over discretized dimensions. With this parameterization, it is possible to both leverage the compositional structure of action spaces during learning, as well as compute maxima over action spaces (approximately). On a simple example task we demonstrate empirically that our method can perform global search, which effectively gets around the local optimization issues that plague DDPG. We apply the technique to off-policy (Q-learning) methods and show that our method can achieve the state-of-the-art for off-policy methods on several continuous control tasks. This paper concerns the worst-case complexity of Gauss-Seidel method for solving a positive semi-definite linear system; or equivalently, that of cyclic coordinate descent (C-CD) for minimizing a convex quadratic function. The known provable complexity of C-CD can be @math times slower than gradient descent (GD) and @math times slower than randomized coordinate descent (R-CD). However, these gaps seem rather puzzling since so far they have not been observed in practice; in fact, C-CD usually converges much faster than GD and sometimes comparable to R-CD. Thus some researchers believe the gaps are due to the weakness of the proof, but not that of the C-CD algorithm itself. In this paper we show that the gaps indeed exist. We prove that there exists an example for which C-CD takes at least @math or @math operations, where @math is the condition number, @math is a well-studied quantity that determines the convergence rate of R-CD, and @math hides the dependency on @math . This implies that C-CD can indeed be @math times slower than GD, and @math times slower than R-CD in the worst case. Our result establishes one of the few examples in continuous optimization that demonstrates a large gap between a deterministic algorithm and its randomized counterpart. Based on the example, we establish several almost tight complexity bounds of C-CD for quadratic problems. One difficulty with the analysis of the constructed example is that the spectral radius of a non-symmetric iteration matrix does not necessarily constitutes a lower bound for the convergence rate. Abstract When the initial distribution and transition rates for a continuous time Markov chain are not known precisely, robust methods are needed to study the evolution of the process in time to avoid judgements based on unwarranted precision. We follow the ideas successfully applied in the study of discrete time model to build a framework of imprecise Markov chains in continuous time. The imprecision in the distributions over the set of states is modelled with upper and lower expectation functionals, which equivalently represent sets of probability distributions. Uncertainty in transitions is modelled with sets of transition rates compatible with available information. The Kolmogorov’s backward equation is then generalised into the form of a generalised differential equation, with generalised derivatives and set valued maps. The upper and lower expectation functionals corresponding to imprecise distributions at given times are determined by the maximal and minimal solutions of these equations. The second part of the paper is devoted to numerical methods for approximating the boundary solutions. The methods are based on discretisation of the time interval. A uniform and adaptive grid discretisations are examined. The latter is computationally much more efficient than the former one, but is not applicable on every interval. Therefore, to achieve maximal efficiency a combination of the methods is used. We present a fully adaptive numerical scheme for evolutionary PDEs in Cartesian geometry based on a second-order finite volume discretization. A multiresolution strategy allows local grid refinement while controlling the approximation error in space. For time discretization we use an explicit Runge-Kutta scheme of second-order with a scale-dependent time step. On the finest scale the size of the time step is imposed by the stability condition of the explicit scheme. On larger scales, the time step can be increased without violating the stability requirement of the explicit scheme. The implementation uses a dynamic tree data structure. Numerical validations for test problems in one space dimension demonstrate the efficiency and accuracy of the local time-stepping scheme with respect to both multiresolution scheme with global time stepping and finite volume scheme on a regular grid. Fully adaptive three-dimensional computations for reaction-diffusion equations illustrate the additional speed-up of the local time stepping for a thermo-diffusive flame instability.