Behaviour cloning is a commonly used strategy for imitation learning and can be extremely effective in constrained domains. However, in cases where the dynamics of an environment may be state dependent and varying, behaviour cloning places a burden on model capacity and the number of demonstrations required. This paper introduces switching density networks, which rely on a categorical reparametrisation for hybrid system identification. This results in a network comprising a classification layer that is followed by a regression layer. We use switching density networks to predict the parameters of hybrid control laws, which are toggled by a switching layer to produce different controller outputs, when conditioned on an input state. This work shows how switching density networks can be used for hybrid system identification in a variety of tasks, successfully identifying the key joint angle goals that make up manipulation tasks, while simultaneously learning image-based goal classifiers and regression networks that predict joint angles from images. We also show that they can cluster the phase space of an inverted pendulum, identifying the balance, spin and pump controllers required to solve this task. Switching density networks can be difficult to train, but we introduce a cross entropy regularisation loss that stabilises training.
Numerous models and approaches @cite_18 have been developed to address the learning problem formulated above. Gaussian mixture models fit using expectation maximisation @cite_31 are widely used for clustering, while their switching state space analog, Gaussian emission hidden Markov models have a long history of application in sequence learning. Although typically fit using the Baum-Welch algorithm @cite_32 (a form of expectation maximisation), variational approaches have also been proposed for a broader class of switching state space models @cite_22 .
@cite_18 @cite_31 @cite_32 @cite_22
1530730921 2049633694 2105594594 2102716594
1. Basic Principles: The Operating Regime Approach 2. Modelling: Fuzzy Set Methods for Local Modelling Identification 3. Modelling of Electrically Stimulated Muscle 4. Process Modelling Using a Functional State Approach 5. Markov Mixtures of Experts 6. Active Learning With Mixture Models 7. Local Learning in Local Model Networks 8. Side Effects of Normalising Basic Functions 9. Control: Heterogeneous Control Laws 10. Local Laguerre Models 11. Multiple Model Adaptive Control 12. H Control Using Multiple Linear Models 13. Synthesis of Fuzzy Control Systems Based on Linear Takagi-Sugeno Fuzzy Models The basic theory of Markov chains has been known to mathematicians and engineers for close to 80 years, but it is only in the past decade that it has been applied explicitly to problems in speech processing. One of the major reasons why speech models, based on Markov chains, have not been developed until recently was the lack of a method for optimizing the parameters of the Markov model to match observed signal patterns. Such a method was proposed in the late 1960's and was immediately applied to speech processing in several research institutions. Continued refinements in the theory and implementation of Markov modelling techniques have greatly enhanced the method, leading to a wide range of applications of these models. It is the purpose of this tutorial paper to give an introduction to the theory of Markov models, and to illustrate how they have been applied to problems in speech recognition. We introduce a new statistical model for time series that iteratively segments data into regimes with approximately linear dynamics and learns the parameters of each of these linear regimes. This model combines and generalizes two of the most widely used stochastic time-series models— hidden Markov models and linear dynamical systems—and is closely related to models that are widely used in the control and econometrics literatures. It can also be derived by extending the mixture of experts neural network (Jacobs, Jordan, Nowlan, & Hinton, 1991) to its fully dynamical version, in which both expert and gating networks are recurrent. Inferring the posterior probabilities of the hidden states of this model is computationally intractable, and therefore the exact expectation maximization (EM) algorithm cannot be applied. However, we present a variational approximation that maximizes a lower bound on the log-likelihood and makes use of both the forward and backward recursions for hidden Markov models and the Kalman filter recursions for linear dynamical systems. We tested the algorithm on artificial data sets and a natural data set of respiration force from a patient with sleep apnea. The results suggest that variational approximations are a viable method for inference and learning in switching state-space models.