Filters in convolutional networks are typically parameterized in a pixel basis, that does not take prior knowledge about the visual world into account. We investigate the generalized notion of frames designed with image properties in mind, as alternatives to this parametrization. We show that frame-based ResNets and Densenets can improve performance on Cifar-10+ consistently, while having additional pleasant properties like steerability. By exploiting these transformation properties explicitly, we arrive at dynamic steerable blocks. They are an extension of residual blocks, that are able to seamlessly transform filters under pre-defined transformations, conditioned on the input at training and inference time. Dynamic steerable blocks learn the degree of invariance from data and locally adapt filters, allowing them to apply a different geometrical variant of the same filter to each location of the feature map. When evaluated on the Berkeley Segmentation contour detection dataset, our approach outperforms all competing approaches that do not utilize pre-training. Our results highlight the benefits of image-based regularization to deep networks.
Steerable frames are a concept established early for signal processing. Initially introduced by @cite_19 , the concept was extended to the Steerable Pyramid by @cite_21 and to a Lie-group formulation by @cite_20 @cite_31 . Further, steerability has recently been related to tight frames, presenting Simoncelli's Steerable Pyramid and multiple other Wavelets arising as a special case of the non-orthogonal Riesz transform @cite_8 . Steerable pyramids have been applied to CNNs as a pre-processing step @cite_4 , but have not yet been learnable. We incorporate steerable frames in CNNs to increase their de facto expressiveness and to allow them to learn their configurations, rather than picking them a priori.
@cite_4 @cite_8 @cite_21 @cite_19 @cite_31 @cite_20
2042583427 2785147712 127298752 1891629850
We introduce a complete parameterization of the family of two-dimensional steerable wavelets that are polar-separable in the Fourier domain under the constraint of self-reversibility. These wavelets are constructed by multiorder generalized Riesz transformation of a primary isotropic bandpass pyramid. The backbone of the transform (pyramid) is characterized by a radial frequency profile function @math , while the directional wavelet components at each scale are encoded by an @math shaping matrix @math , where @math is the number of wavelet channels and @math the order of the Riesz transform. We provide general conditions on @math and @math for the underlying wavelet system to form a tight frame of @math (with a redundancy factor @math ). The proposed framework ensures that the wavelets are steerable and provides new degrees of freedom (shaping matrix @math ) that can be exploited for designing specific wavelet systems. It encompasses many known transforms as part... Convolutional Neural Networks (CNN) have been successfully applied to autonomous driving tasks, many in an end-to-end manner. Previous end-to-end steering control methods take an image or an image sequence as the input and directly predict the steering angle with CNN. Although single task learning on steering angles has reported good performances, the steering angle alone is not sufficient for vehicle control. In this work, we propose a multi-task learning framework to predict the steering angle and speed control simultaneously in an end-to-end manner. Since it is nontrivial to predict accurate speed values with only visual inputs, we first propose a network to predict discrete speed commands and steering angles with image sequences. Moreover, we propose a multi-modal multi-task network to predict speed values and steering angles by taking previous feedback speeds and visual recordings as inputs. Experiments are conducted on the public Udacity dataset and a newly collected SAIC dataset. Results show that the proposed model predicts steering angles and speed values accurately. Furthermore, we improve the failure data synthesis methods to solve the problem of error accumulation in real road tests. A function is called steerable if transformed versions of the function can be expressed using linear combinations of a fixed set of basis functions. Steerability is a very general and often desirable property. As a result, steerable functions have recently been applied to an assortment of problems in image processing, computer vision, and computer graphics. In this dissertation, we propose a framework, based on Lie group theory, for studying and constructing functions steerable under any smooth transformation group. We argue that Lie theory is the appropriate mathematical tool for analyzing the properties of these steerable functions. This position is supported by the fact that all existing analytical approaches to steerability can be consistently explained within the framework. The design of a suitable set of basis functions given any arbitrary steerable function is one of the main problems concerning steerable functions. To this end, we have developed two very different algorithms. The first algorithm is a symbolic method that can be implemented in any symbolic package. This algorithm derives the minimal set of basis functions automatically given an arbitrary steerable function. The second algorithm addresses two practical considerations: approximate steerability and local steerability. In practice, functions that need to be steered might not be steerable with a finite number of basis functions. Moreover, it is often the case that only a small subset of transformations within the group of transformations needs to be considered. In response to these two concerns, the second algorithm computes the optimal set of k basis functions to steer an arbitrary function under a subset of the group of transformations. Lastly, we demonstrate the usefulness of steerable functions in a variety of applications. In particular, we present five applications that use steerable functions: (1) continuum approximation in modeling human vision (a method of approximating an infinite number of interacting mechanisms in a model), (2) the design of optimal steerable filters for gradient-based motion estimation, (3) efficient linear re-rendering of synthetic scenes under changes in illumination, (4) the construction of invariants from steerable filters, and (5) the application of steerable functions to discrete sets of points and lines. Our aim is to optimize wavelet-based feature extraction for differentiating between the classical versus atypical pattern of usual interstitial pneumonia (UIP) in volumetric CT. Our proposal is to act on the bandwidth of steerable wavelets while maintaining their tight frame property. To that end, we designed a family of maximally localized wavelet pyramids in 3-D for a continuously adjustable radial bandwidth [Ω,π], Ω G [π 4, π 2]. The proposed wavelets are coupled with a rotation-covariant directional operator based on the Riesz transform, which provides characterizations of the organization image directions independently from their local orientations. The influence of the wavelet bandwidth on the classification performance was found to be large with area under the receiver operating characteristic curve (AUC) values in [0.784,0.921]. This demonstrated the importance of finding the minimum spatial support of the wavelet required to leverage the wealth of morphological tissue properties in the vicinity of the lung boundaries.