Representation Learning: A Review and New Perspectives <s> Alternatives to Pooling <s> This paper constructs translation invariant operators on L2(R^d), which are Lipschitz continuous to the action of diffeomorphisms. A scattering propagator is a path ordered product of non-linear and non-commuting operators, each of which computes the modulus of a wavelet transform. A local integration defines a windowed scattering transform, which is proved to be Lipschitz continuous to the action of diffeomorphisms. As the window size increases, it converges to a wavelet scattering transform which is translation invariant. Scattering coefficients also provide representations of stationary processes. Expected values depend upon high order moments and can discriminate processes having the same power spectrum. Scattering operators are extended on L2 (G), where G is a compact Lie group, and are invariant under the action of G. Combining a scattering on L2(R^d) and on Ld (SO(d)) defines a translation and rotation invariant scattering on L2(R^d). <s> BIB001 </s> Representation Learning: A Review and New Perspectives <s> Alternatives to Pooling <s> A scattering vector is a local descriptor including multiscale and multi-direction co-occurrence information. It is computed with a cascade of wavelet decompositions and complex modulus. This scattering representation is locally translation invariant and linearizes deformations. A supervised classification algorithm is computed with a PCA model selection on scattering vectors. State of the art results are obtained for handwritten digit recognition and texture classification. <s> BIB002
Alternatively, one can also use explicit knowledge of the expected invariants expressed mathematically to define transformations that are robust to a known family of input deformations, using the so-called scattering operators BIB001 , BIB002 , which can be computed in a way interestingly analogous to deep convolutional networks and wavelets. Like convolutional networks, the scattering operators alternate two types of operations: convolution and pooling (as a norm). Unlike convolutional networks, the proposed approach keeps at each level all of the information about the input (in a way that can be inverted) and automatically yields a very sparse (but very high dimensional) representation. Another difference is that the filters are not learned but instead are set so as to guarantee that a priori specified invariances are robustly achieved. Just a few levels were sufficient to achieve impressive results on several benchmark datasets.