Independent component analysis (ICA) aims at decomposing an observed random vector into statistically independent variables. Deflation-based implementations, such as the popular one-unit FastICA algorithm and its variants, extract the independent components one after another. A novel method for deflationary ICA, referred to as RobustICA, is put forward in this paper. This simple technique consists of performing exact line search optimization of the kurtosis contrast function. The step size leading to the global maximum of the contrast along the search direction is found among the roots of a fourth-degree polynomial. This polynomial rooting can be performed algebraically, and thus at low cost, at each iteration. Among other practical benefits, RobustICA can avoid prewhitening and deals with real- and complex-valued mixtures of possibly noncircular sources alike. The absence of prewhitening improves asymptotic performance. The algorithm is robust to local extrema and shows a very high convergence speed in terms of the computational cost required to reach a given source extraction quality, particularly for short data records. These features are demonstrated by a comparative numerical analysis on synthetic data. RobustICA's capabilities in processing real-world data involving noncircular complex strongly super-Gaussian sources are illustrated by the biomedical problem of atrial activity (AA) extraction in atrial fibrillation (AF) electrocardiograms (ECGs), where it outperforms an alternative ICA-based technique.
Amari @cite_15 @cite_56 puts forward adaptive rules for learning the step size in neural algorithms for BSS ICA, more pertinent in the context of the present work. The idea is to make the step size depend on the gradient norm, in order to obtain a fast evolution at the beginning of the iterations and then a decreasing misadjustment as a stationary point is reached. These step-size learning rules, in turn, include other learning coefficients which must be set appropriately. Although the resulting algorithms are said to be robust to the choice of these coefficients, their optimal selection remains application dependent. Other guidelines for choosing the step size in natural gradient algorithms are given in @cite_60 , but are merely based on local stability conditions. In a non-linear mixing setup, Khor and co-workers put forward a fuzzy logic approach to control the learning rate of a separation algorithm based on the natural gradient @cite_13 .
@cite_60 @cite_15 @cite_13 @cite_56
2158035059 1970789124 2088551704
This paper reports a study on the problem of the blind simultaneous extraction of specific groups of independent components from a linear mixture. This paper first presents a general overview and unification of several information theoretic criteria for the extraction of a single independent component. Then, our contribution fills the theoretical gap that exists between extraction and separation by presenting tools that extend these criteria to allow the simultaneous blind extraction of subsets with an arbitrary number of independent components. In addition, we analyze a family of learning algorithms based on Stiefel manifolds and the natural gradient ascent, present the nonlinear optimal activations (score) functions, and provide new or extended local stability conditions. Finally, we illustrate the performance and features of the proposed approach by computer-simulation experiments. When a parameter space has a certain underlying structure, the ordinary gradient of a function does not represent its steepest direction, but the natural gradient does. Information geometry is used for calculating the natural gradients in the parameter space of perceptrons, the space of matrices (for blind source separation), and the space of linear dynamical systems (for blind source deconvolution). The dynamical behavior of natural gradient online learning is analyzed and is proved to be Fisher efficient, implying that it has asymptotically the same performance as the optimal batch estimation of parameters. This suggests that the plateau phenomenon, which appears in the backpropagation learning algorithm of multilayer perceptrons, might disappear or might not be so serious when the natural gradient is used. An adaptive method of updating the learning rate is proposed and analyzed. This paper proposes a new nonlinear blind source separation algorithm with hybridisation of fuzzy logic based learning rate control and simulated annealing to improve the global solution search. Benefits of fuzzy systems and simulated annealing are incorporated into a multilayer perceptron network. Fuzzy logic control allows adjustments of learning rate to enhance the rate of convergence of the algorithm. Simulated annealing is implemented to avoid the algorithm becoming trapped in local minima. A simple and computationally efficient method for controlling learning rate and ensuring a global solution is proposed. The performance of the proposed algorithm in terms of convergence of entropy, is studied alongside other techniques of learning rate adaptation. Simulations show that the proposed nonlinear algorithm outperforms other existing nonlinear algorithms based on fixed learning rates.