Behaviour cloning is a commonly used strategy for imitation learning and can be extremely effective in constrained domains. However, in cases where the dynamics of an environment may be state dependent and varying, behaviour cloning places a burden on model capacity and the number of demonstrations required. This paper introduces switching density networks, which rely on a categorical reparametrisation for hybrid system identification. This results in a network comprising a classification layer that is followed by a regression layer. We use switching density networks to predict the parameters of hybrid control laws, which are toggled by a switching layer to produce different controller outputs, when conditioned on an input state. This work shows how switching density networks can be used for hybrid system identification in a variety of tasks, successfully identifying the key joint angle goals that make up manipulation tasks, while simultaneously learning image-based goal classifiers and regression networks that predict joint angles from images. We also show that they can cluster the phase space of an inverted pendulum, identifying the balance, spin and pump controllers required to solve this task. Switching density networks can be difficult to train, but we introduce a cross entropy regularisation loss that stabilises training.
Switching density networks are closely related to mixture density networks @cite_10 , a family of neural network constructed using @math output distributions. In the Gaussian mixture case, MDNs fit a weighted combination of Gaussian distributions, using mean @math , variance @math and normalised weight parameters @math , which are predicted using a neural network. Unfortunately, there is no direct link between weight components and mean or variance components, so mixture density networks often learn seemingly arbitrary connections. We illustrate this experimentally in Section , showing that an MDN trained to predict manipulator joint angles will use only a single mixture component for completely different joint angle predictions, somewhat unintuitively learning to change the mean and variance parameters instead of toggling between mixture components. This occurs because no structure forces mixture consistency in the network.
@cite_10
2805701896 1579853615 2168227362 2589209256
Artificial Neural Networks (ANNs) are increasingly used in modeling and prediction due to advances in deep learning architecture and training. As ANNs become a viable option for making state or measurement predictions in navigation algorithms, special care must be taken in order to determine the confidence (covariance) of the predictions. A special kind of ANN called a Mixture Density Network (MDN) approximates the prediction distribution with a mixture of distributions such as Gaussian distributions. This allows the MDN to output a full Probability Distribution Function (PDF) rather than just a mean estimate. The prediction mixtures can also be converted into other forms for processing by a filter. In this paper, MDN are trained to model a measurement equation which converts measurements to state estimates represented as a Gaussian Mixture Model (GMM). The GMM distributions are simplified into a single Gaussian distribution for each measurement for inclusion into an EKF. An experiment was conducted using magnetic field measurements and Very Low Frequency (VLF) signals to estimate position using an EKF with measurements from a MDN. This method shows a performance improvement using the MDN covariance estimate to determine measurement noise when compared to using static estimates for measurement noise. Thus, the MDN output allows the modeling and prediction power of ANN to be used in standard navigation filters with minimal changes to the filter. Minimization of a sum-of-squares or cross-entropy error function leads to network outputs which approximate the conditional averages of the target data, conditioned on the input vector. For classifications problems, with a suitably chosen target coding scheme, these averages represent the posterior probabilities of class membership, and so can be regarded as optimal. For problems involving the prediction of continuous variables, however, the conditional averages provide only a very limited description of the properties of the target variables. This is particularly true for problems in which the mapping to be learned is multi-valued, as often arises in the solution of inverse problems, since the average of several correct target values is not necessarily itself a correct value. In order to obtain a complete description of the data, for the purposes of predicting the outputs corresponding to new input vectors, we must model the conditional probability distribution of the target data, again conditioned on the input vector. In this paper we introduce a new class of network models obtained by combining a conventional neural network with a mixture density model. The complete system is called a Mixture Density Network, and can in principle represent arbitrary conditional probability distributions in the same way that a conventional neural network can represent arbitrary functions. We demonstrate the effectiveness of Mixture Density Networks using both a toy problem and a problem involving robot inverse kinematics. Gaussian mixtures (or so-called radial basis function networks) for density estimation provide a natural counterpart to sigmoidal neural networks for function fitting and approximation. In both cases, it is possible to give simple expressions for the iterative improvement of performance as components of the network are introduced one at a time. In particular, for mixture density estimation we show that a k-component mixture estimated by maximum likelihood (or by an iterative likelihood improvement that we introduce) achieves log-likelihood within order 1 k of the log-likelihood achievable by any convex combination. Consequences for approximation and estimation using Kullback-Leibler risk are also given. A Minimum Description Length principle selects the optimal number of components k that minimizes the risk bound. As deep neural networks (DNNs) are applied to increasingly challenging problems, they will need to be able to represent their own uncertainty. Modelling uncertainty is one of the key features of Bayesian methods. Bayesian DNNs that use dropout-based variational distributions and scale to complex tasks have recently been proposed. We evaluate Bayesian DNNs trained with Bernoulli or Gaussian multiplicative masking of either the units (dropout) or the weights (dropconnect). We compare these Bayesian DNNs ability to represent their uncertainty about their outputs through sampling during inference. We tested the calibration of these Bayesian fully connected and convolutional DNNs on two visual inference tasks (MNIST and CIFAR-10). By adding different levels of Gaussian noise to the test images, we assessed how these DNNs represented their uncertainty about regions of input space not covered by the training set. These Bayesian DNNs represented their own uncertainty more accurately than traditional DNNs with a softmax output. We find that sampling of weights, whether Gaussian or Bernoulli, led to more accurate representation of uncertainty compared to sampling of units. However, sampling units using either Gaussian or Bernoulli dropout led to increased convolutional neural network (CNN) classification accuracy. Based on these findings we use both Bernoulli dropout and Gaussian dropconnect concurrently, which approximates the use of a spike-and-slab variational distribution. We find that networks with spike-and-slab sampling combine the advantages of the other methods: they classify with high accuracy and robustly represent the uncertainty of their classifications for all tested architectures.