A pseudoinverse incremental algorithm for fast training deep neural networks with application to spectra pattern recognition
Deep learning scheme has received significant attention during these years, particularly as a way of building hierarchical representations from unlabeled data for a variety of signal and information processing tasks. However, deep neural networks suffer from slow learning speed since most used training algorithms are based on variations of the gradient descent algorithms which require iterative optimization and thus are time consuming. In addition, a series of control parameters need to be specified empirically which lacks of the theoretical guidance, and current learning algorithms for deep networks are not very suitable to incremental learning scenario. To address these issues, we propose a fast learning scheme in this paper. The basic idea of our approach is to pre train basic units such as auto encoders of the deep architecture in an analytical way without any iterative optimization procedure. This scheme is also extended to an incremental learning version. The experimental result shows the superiority of our approach over the state of the art gradient descent based algorithms. To demonstrate the impact of our algorithm on complicated real world applications, we give an example of its performance in astronomical spectra pattern recognition.
2016 International Joint Conference on Neural Networks (IJCNN)