This paper discusses a novel initialization algorithm for the estimation of nonlinear state-space models. Good initial values for the model parameters are obtained by identifying separately the linear dynamics and the nonlinear terms in the model. In particular, the nonlinear dynamic problem is transformed into an approximate static formulation, and simple regression methods are applied to obtain the solution in a fast and efficient way. The proposed method is validated by means of two measurement examples: the Wiener-Hammerstein benchmark problem and the identification of a crystal detector.
More recently, a combination of multilayer perceptron NNs, the EM algorithm and particle smoothing is suggested in @cite_3 for joint parameter and state estimation.
@cite_3
2122263489 2094887027 2067472975 2135354436
Identification of nonlinear state space models when no information is available from the state transition or output model has played an important role in the recent research. In this paper, we propose a new approach for modeling a discrete time nonlinear state space system with a multi layer perceptron (MLP) neural network. The expectation maximization (EM) algorithm is used for joint parameter and state estimation of the proposed structure where the particle smoothing algorithm will be applied for estimating hidden states. Because of the non-affine structure of MLP networks compared with some other models such as radial basis functions, the gradient method is used at the M phase of the EM algorithm for parameter and noise estimation. Simulation studies show the superiority and fast convergence of our proposed structure in identification of nonlinear state space models. We propose a wavelet multiscale decomposition-based autoregressive approach for the prediction of 1-h ahead load based on historical electricity load data. This approach is based on a multiple resolution decomposition of the signal using the non-decimated or redundant Haar a trous wavelet transform whose advantage is taking into account the asymmetric nature of the time-varying data. There is an additional computational advantage in that there is no need to recompute the wavelet transform (wavelet coefficients) of the full signal if the electricity data (time series) is regularly updated. We assess results produced by this multiscale autoregressive (MAR) method, in both linear and non-linear variants, with single resolution autoregression (AR), multilayer perceptron (MLP), Elman recurrent neural network (ERN) and the general regression neural network (GRNN) models. Results are based on the New South Wales (Australia) electricity load data that is provided by the National Electricity Market Management Company (NEMMCO). A neural network particle finding algorithm and a new four-frame predictive tracking algorithm are proposed for three-dimensional Lagrangian particle tracking (LPT). A quantitative comparison of these and other algorithms commonly used in three-dimensional LPT is presented. Weighted averaging, one-dimensional and two-dimensional Gaussian fitting, and the neural network scheme are considered for determining particle centers in digital camera images. When the signal to noise ratio is high, the one-dimensional Gaussian estimation scheme is shown to achieve a good combination of accuracy and efficiency, while the neural network approach provides greater accuracy when the images are noisy. The effect of camera placement on both the yield and accuracy of three-dimensional particle positions is investigated, and it is shown that at least one camera must be positioned at a large angle with respect to the other cameras to minimize errors. Finally, the problem of tracking particles in time is studied. The nearest neighbor algorithm is compared with a three-frame predictive algorithm and two four-frame algorithms. These four algorithms are applied to particle tracks generated by direct numerical simulation both with and without a method to resolve tracking conflicts. The new four-frame predictive algorithm with no conflict resolution is shown to give the best performance. Finally, the best algorithms are verified to work in a real experimental environment. Multilayer perceptrons (MLPs) or neural networks are popular models used for nonlinear regression and classification tasks. As regressors, MLPs model the conditional distribution of the predictor variables Y given the input variables X. However, this predictive distribution is assumed to be unimodal (e.g. Gaussian). For tasks involving structured prediction, the conditional distribution should be multi-modal, resulting in one-to-many mappings. By using stochastic hidden variables rather than deterministic ones, Sigmoid Belief Nets (SBNs) can induce a rich multimodal distribution in the output space. However, previously proposed learning algorithms for SBNs are not efficient and unsuitable for modeling real-valued data. In this paper, we propose a stochastic feedforward network with hidden layers composed of both deterministic and stochastic variables. A new Generalized EM training procedure using importance sampling allows us to efficiently learn complicated conditional distributions. Our model achieves superior performance on synthetic and facial expressions datasets compared to conditional Restricted Boltzmann Machines and Mixture Density Networks. In addition, the latent features of our model improves classification and can learn to generate colorful textures of objects.