System identification is the process of building a mathematical model of an unknown system from measurements of its inputs and outputs. It is a key step for model-based control, estimator design, and output prediction. This work presents an algorithm for non-linear offline system identification from partial observations, i.e. situations in which the system's full-state is not directly observable. The algorithm presented, called SISL, iteratively infers the system's full state through non-linear optimization and then updates the model parameters. We test our algorithm on a simulated system of coupled Lorenz attractors, showing our algorithm's ability to identify high-dimensional systems that prove intractable for particle-based approaches. We also use SISL to identify the dynamics of an aerobatic helicopter. By augmenting the state with unobserved fluid states, we learn a model that predicts the acceleration of the helicopter better than state-of-the-art approaches. The performance of controllers and state-estimators for non-linear systems depends heavily on the quality of the model of system dynamics (Hou & Wang, 2013) . System-identification tackles the problem of learning or calibrating dynamics models from data (Ljung, 1999) , which is often a timehistory of observations of the system and control inputs. In this work, we address the problem of learning dynamics models of partially observed, high-dimensional non-linear systems. That is, we consider situations in which the system's state cannot be inferred from a single observation, but instead requires inference over a time-series of observations. The problem of identifying systems from partial observations arises in many robotics domains (Punjani & Abbeel, 2015; Cory & Tedrake, 2008; Ordonez et al., 2017) . Though we often have direct measurements of a robot's pose and velocity, in many cases we cannot directly observe relevant quantities such as the temperature of actuators or the state the environment around the robot. Consider learning a dynamics model for an aerobatic helicopter. Abbeel et al. (2010) attempted to map only the helicopter's pose and velocity to its acceleration and they found their model to be inaccurate when predicting aggressive maneuvers. They posited that the substantial airflow generated by the helicopter affected the dynamics. Since it is often impossible to directly measure the state of the airflow around a vehicle, identification must be performed in a partially observed setting. System-identification is a mature field with a rich history (Ljung, 1999; 2010) . Various techniques can be classified by whether they apply to linear or non-linear systems, with partially or fully observed states. Additionally, techniques are applied in an online or batch-offline setting. This work presents an approach to offline identification of non-linear and partially observed systems. When a system is fully observed, i.e. its full state is observed but corrupted by noise, a set of techniques called equation-error methods are typically employed (Åström & Eykhoff, 1971) . In such cases, we can consider observations as independent, and minimize the error between the observed statederivatives and those predicted by the model given the control input and observed states. In partially observed settings, merely knowing the current input is insufficient to accurately predict the observation. Several black-box approaches exist to predict observations from time-series of inputs. Autoregressive approaches directly map a time-history of past inputs to observations (Billings, 2013) . Recurrent neural networks (Bailer-Jones et al., 1998; Zimmermann & Neuneier, 2000) and subspace-identification methods (Van Overschee & De Moor, 1994) can also be used to learn blackbox dynamical systems from this data. However, in many cases prior knowledge can be used to specify structured, parameterized models of the system (Gupta et al., 2019) . Such models can be trained with less data and used with a wider array of control and state-estimation techniques than non-linear black-box models (Gupta et al., 2019; Lutter et al., 2019b; a) . Techniques used to identify partially observed structured models are often based on Expectation-Maximization (EM) (Dempster et al., 1977; Schön et al., 2011; Kantas et al., 2015; Ghahramani & Roweis, 1999 ). An alternating procedure is performed in which a smoothing step uses the current system dynamics estimate to infer the distribution over state-trajectories, and is followed by a learning step that uses this distribution to update the system dynamics estimate. In the non-linear or non-Gaussian case, it is typically not possible to analytically characterize the distribution over trajectories, and thus methods based on Sequential Monte-Carlo such as Particle Smoothing (PS) (Schön et al., 2011; Kantas et al., 2015) , or Extended Kalman Smoothing (EKS) (Ghahramani & Roweis, 1999) are employed in the E-step. Though considered state-of-theart for this problem, both methods become intractable in high-dimensional state spaces. PS suffers from the curse of dimensionality, requiring an intractably large number of particles if the state space is high-dimensional (Snyder et al., 2008; Kantas et al., 2015) , and an M-step that can be quadratic in complexity with respect to the number of particles (Schön et al., 2011) . EKS-based methods are fast during the E-step, but the M-step requires approximations to integrate out state uncertainty, such as fitting non-linearities with Radial Basis Function approximators, and scales poorly with the dimension of the state-space (Ghahramani & Roweis, 1999) . In this work, we present a system-identification algorithm that is suited for high-dimensional, nonlinear, and partially observed systems. By assuming that the systems are close to deterministic, as is often the case in robotics, we approximate the distribution over unobserved states using only their maximum-likelihood (ML) point-estimate. Our algorithm, called SISL (System-identification via Iterative Smoothing and Learning) performs the following two steps until convergence: • In the smoothing or E-step, we use non-linear programming to tractably find the ML pointestimate of the unobserved states. • In the learning or M-step, we use the estimate of unobserved states to improve the estimate of system parameters. The idea to use an ML point-estimate in lieu of the distribution over unobserved states in the EM procedure's E-step is not new, and, in general, does not guarantee monotonic convergence to a local optimum (Celeux & Govaert, 1992) . However, such an approximation is equivalent to regular EM if the ML point-estimate is the only instance of unobserved variables with non-negligible probability (Celeux & Govaert, 1992; Neal & Hinton, 1998) . We apply this idea to the problem of system-identification for nearly deterministic systems, in which ML point-estimates can serve as surrogates for the true distribution over unobserved state-trajectories. The primary contribution of this work is an algorithm for identifying non-linear, partially observed systems that is able to scale to high-dimensional problems. In Section 2, we specify the assumptions underpinning our algorithm and discuss the computational methodology for using it. In Section 3, we empirically demonstrate that it is able to identify the parameters of a high-dimensional system of coupled Lorenz attractors, a problem that proves intractable for particle-based methods. We also demonstrate our algorithm on the problem of identifying the dynamics of an aerobatic helicopter, and compare against various approaches including the state-of-the-art approach (Punjani & Abbeel, 2015) . This paper presented an algorithm for system identification of non-linear systems given partial state observations. The algorithm optimizes system parameters given a time history of observations by iteratively finding the most likely state-history, and then using it to optimize the system parameters. The approach is particularly well suited for high-dimensional and nearly deterministic problems. In simulated experiments on a partially observed system of coupled Lorenz attractors, we showed that our algorithm can perform identification on a problem that particle-based EM methods are fundamentally ill-suited for. We also validated that our algorithm is an effective replacement for identification methods based on EM if the system is close to deterministic, but can yield biased parameter estimates if it is not. We then used our algorithm to model the time-varying hiddenstates that affect the dynamics of an aerobatic helicopter. Our approach outperforms state-of-the-art methods because it is able to fit large non-linear models to unobserved states. We aim to apply our algorithm to system identification problems in a number of domains. There has recently been interest in characterizing the dynamics of aircraft with high aspect ratios, for which the difficult-to-observe bending modes substantially impact dynamics. Additionally, the inability to measure friction forces in dynamic interactions involving contact typically stands in the way of system identification, and thus requires algorithms that are capable of identification under partial observation. A APPENDIX
This work presents a scalable algorithm for non-linear offline system identification from partial observations.
{"src": "scitldr-AIC_train", "paper_id": "B1gR3ANFPS"}