A Survey on the New Generation of Deep Learning in Image Processing <s> 4) ML-CSC <s> In this paper, we present two fast sparse approximation schemes for least squares support vector machine (LS-SVM), named FSALS-SVM and PFSALS-SVM, to overcome the limitation of LS-SVM that it is not applicable to large data sets and to improve test speed. FSALS-SVM iteratively builds the decision function by adding one basis function from a kernel-based dictionary at one time. The process is terminated by using a flexible and stable epsilon insensitive stopping criterion. A probabilistic speedup scheme is employed to further improve the speed of FSALS-SVM and the resulting classifier is named PFSALS-SVM. Our algorithms are of two compelling features: low complexity and sparse solution. Experiments on benchmark data sets show that our algorithms obtain sparse classifiers at a rather low cost without sacrificing the generalization performance <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 4) ML-CSC <s> The recently proposed Multi-Layer Convolutional Sparse Coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of Convolutional Neural Networks (CNNs). Under this framework, the computation of the forward pass in a CNN is equivalent to a pursuit algorithm aiming to estimate the nested sparse representation vectors -- or feature maps -- from a given input signal. Despite having served as a pivotal connection between CNNs and sparse modeling, a deeper understanding of the ML-CSC is still lacking: there are no pursuit algorithms that can serve this model exactly, nor are there conditions to guarantee a non-empty model. While one can easily obtain signals that approximately satisfy the ML-CSC constraints, it remains unclear how to simply sample from the model and, more importantly, how one can train the convolutional filters from real data. In this work, we propose a sound pursuit algorithm for the ML-CSC model by adopting a projection approach. We provide new and improved bounds on the stability of the solution of such pursuit and we analyze different practical alternatives to implement this in practice. We show that the training of the filters is essential to allow for non-trivial signals in the model, and we derive an online algorithm to learn the dictionaries from real data, effectively resulting in cascaded sparse convolutional layers. Last, but not least, we demonstrate the applicability of the ML-CSC model for several applications in an unsupervised setting, providing competitive results. Our work represents a bridge between matrix factorization, sparse dictionary learning and sparse auto-encoders, and we analyze these connections in detail. <s> BIB002
ML-CSC BIB002 , which consists of a cascade of convolutional sparse layers, can provides a new interpretation of CNNs. First, we give a concrete example for better understanding CSC model. assume x ∈ R 25 admits a decomposition as Dα, where α ∈ R (25·m) is sparse and the dictionary D ∈ R 25×(25·m) has a convolutional structure. Precisely, this dictionary consists of m local n-dimensional filters at every possible location, where m denotes the width of stripe for convolutional structure. For example, m = 2 and n = 5, then CSC model can be shown in Fig.22 , Further, each jth patch P j x ∈ R n (&R 5 ) from the signal x can be expressed in terms of a shift-invariant local model corresponding to a stripe from the global sparse vector, S j α ∈ R (2n−1)m (&R 18 ), where P j can be viewed as patch extraction, S j is a patch extractor in transform space. In the context of CSC, the sparsity of the representation is better captured through the l 0,∞ pseudo-norm. Formally, Given a convolutional dictionary of appropriate dimension, a signal x admits a representation in terms of the CSC model if satisfy, where K ∈ N is the sparsity degree. We can expand CSC model to ML-CSC, that is, given a set of convolutional dictionaries {D l } L l=1 of appropriate dimensions, a signal x admits a representation in terms of the ML-CSC model if satisfy, where K l is the sparsity degree, l = 1, 2, · · · , L, L is the number of layers. Interestingly, the ML-CSC can interpret as a special case of the CSC model. Finally, these sparse coefficients α l can be solved by the deep coding algorithm BIB001 . 