Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism into a computational model remains elusive. Encouraged by a new discovery (CVPR 2016) demonstrating extremely high inter-person accuracy of human perceived symmetries in the wild, we have created the first deep-learning neural network for reflection and rotation symmetry detection (Sym-NET), trained on photos from MS-COCO (Common Object in COntext) dataset with nearly 11K symmetry-labels from more than 400 human observers. We employ novel methods to convert discrete human labels into symmetry heatmaps, capture symmetry densely in an image and quantitatively evaluate Sym-NET against multiple existing computer vision algorithms. Using the symmetry competition testsets from CVPR 2013 and unseen MS-COCO photos, Sym-NET comes out as the winner with significantly superior performance over all other competitors. Beyond mathematically well-defined symmetries on a plane, Sym-NET demonstrates abilities to identify viewpoint-varied 3D symmetries, partially occluded symmetrical objects and symmetries at a semantic level.
Different from recent efforts in the deep learning CNN community where researchers are seeking networks that are rotation reflection or affine invariant to input images @cite_23 @cite_68 @cite_16 @cite_24 , our work explicitly acknowledges (near) reflection and rotation symmetries in the raw data regardless of the transformations applied on the input images. To the best of our knowledge, there have been no deep learning networks trained on human symmetry labels for automated reflection and rotation symmetry detections.
@cite_24 @cite_68 @cite_23 @cite_16
2951320766 2951770173 2340427832
We present a novel and effective algorithm for affinely skewed rotation symmetry group detection from real-world images. We define a complete skewed rotation symmetry detection problem as discovering five independent properties of a skewed rotation symmetry group: 1) the center of rotation, 2) the affine deformation, 3) the type of the symmetry group, 4) the cardinality of the symmetry group, and 5) the supporting region of the symmetry group in the image. We propose a frieze-expansion (FE) method that transforms rotation symmetry group detection into a simple, 1D translation symmetry detection problem. We define and construct a pair of rotational symmetry saliency maps, complemented by a local feature method. Frequency analysis, using Discrete Fourier Transform (DFT), is applied to the frieze-expansion patterns (FEPs) to uncover the types (cyclic, dihedral, and O(2)), the cardinalities, and the corresponding supporting regions, concentric or otherwise, of multiple rotation symmetry groups in an image. The phase information of the FEP is used to rectify affinely skewed rotation symmetry groups. Our result advances the state of the art in symmetry detection by offering a unique combination of region-based, feature-based, and frequency-based approaches. Experimental results on 170 synthetic and natural images demonstrate superior performance of our rotation symmetry detection algorithm over existing methods. We present an algorithm that detects rotational and reflectional symmetries of two-dimensional objects. Both symmetry types are effectively detected and analyzed using the angular correlation (AC), which measures the correlation between images in the angular direction. The AC is accurately computed using the pseudopolar Fourier transform, which rapidly computes the Fourier transform of an image on a near-polar grid. We prove that the AC of symmetric images is a periodic signal whose frequency is related to the order of the symmetry. This frequency is recovered via spectrum estimation, which is a proven technique in signal processing with a variety of efficient solutions. We also provide a novel approach for finding the center of symmetry and demonstrate the applicability of our scheme to the analysis of real images We propose a new model for digital pathology segmentation, based on the observation that histopathology images are inherently symmetric under rotation and reflection. Utilizing recent findings on rotation equivariant CNNs, the proposed model leverages these symmetries in a principled manner. We present a visual analysis showing improved stability on predictions, and demonstrate that exploiting rotation equivariance significantly improves tumor detection performance on a challenging lymph node metastases dataset. We further present a novel derived dataset to enable principled comparison of machine learning models, in combination with an initial benchmark. Through this dataset, the task of histopathology diagnosis becomes accessible as a challenging benchmark for fundamental machine learning research. We present a novel and effective algorithm for rotation symmetry group detection from real-world images. We propose a frieze-expansion method that transforms rotation symmetry group detection into a simple translation symmetry detection problem. We define and construct a dense symmetry strength map from a given image, and search for potential rotational symmetry centers automatically. Frequency analysis, using discrete Fourier transform (DFT), is applied to the frieze-expansion patterns to uncover the types and the cardinality of multiple rotation symmetry groups in an image, concentric or otherwise. Furthermore, our detection algorithm can discriminate discrete versus continuous and cyclic versus dihedral symmetry groups, and identify the corresponding supporting regions in the image. Experimental results on over 80 synthetic and natural images demonstrate superior performance of our rotation detection algorithm in accuracy and in speed over the state of the art rotation detection algorithms. Symmetry is a pervasive phenomenon presenting itself in all forms and scales in natural and manmade environments. Its detection plays an essential role at all levels of human as well as machine perception. The recent resurging interest in computational symmetry for computer vision and computer graphics applications has motivated us to conduct a US NSF funded symmetry detection algorithm competition as a workshop affiliated with the Computer Vision and Pattern Recognition (CVPR) Conference, 2013. This competition sets a more complete benchmark for computer vision symmetry detection algorithms. In this report we explain the evaluation metric and the automatic execution of the evaluation workflow. We also present and analyze the algorithms submitted, and show their results on three test sets of real world images depicting reflection, rotation and translation symmetries respectively. This competition establishes a performance baseline for future work on symmetry detection. Natural images often exhibit symmetries that should be taken into account when editing them. In this paper we present Nautilus --- a method for automatically identifying symmetric regions in an image along with their corresponding symmetry transformations. We compute dense local similarity symmetry transformations using a novel variant of the Generalised PatchMatch algorithm that uses Metropolis-Hastings sampling. We combine and refine these local symmetries using an extended Lucas-Kanade algorithm to compute regional transformations and their spatial extents. Our approach produces dense estimates of complex symmetries that are combinations of translation, rotation, scale, and reflection under perspective distortion. This enables a number of automatic symmetry-aware image editing applications including inpainting, rectification, beautification, and segmentation, and we demonstrate state-of-the-art applications for each of them. We present a spectral approach for detecting and analyzing rotational and reflectional symmetries in n-dimensions. Our main contribution is the derivation of a symmetry detection and analysis scheme for sets of points IRn and its extension to image analysis by way of local features. Each object is represented by a set of points S ∈ IRn, where the symmetry is manifested by the multiple self-alignments of S . The alignment problem is formulated as a quadratic binary optimization problem, with an efficient solution via spectral relaxation. For symmetric objects, this results in a multiplicity of eigenvalues whose corresponding eigenvectors allow the detection and analysis of both types of symmetry. We improve the scheme's robustness by incorporating geometrical constraints into the spectral analysis. Our approach is experimentally verified by applying it to 2D and 3D synthetic objects as well as real images. Reflection symmetry detection receives increasing attentions in recent years. The state-of-the-art algorithms mainly use the matching of intensity-based features (such as the SIFT) within a single image to find symmetry axes. This paper proposes a novel approach by establishing the correspondence of locally affine invariant edge-based features, which are superior to the intensity based in the aspects that it is insensitive to illumination variations, and applicable to textureless objects. The locally affine invariance is achieved by simple linear algebra for efficient and robust computations, making the algorithm suitable for detections under object distortions like perspective projection. Commonly used edge detectors and a voting process are, respectively, used before and after the edge description and matching steps to form a complete reflection detection pipeline. Experiments are performed using synthetic and real-world images with both multiple and single reflection symmetry axis. The test results are compared with existing algorithms to validate the proposed method. Symmetry, as one of the key components of Gestalt theory, provides an important mid-level cue that serves as input to higher visual processes such as segmentation. In this work, we propose a complete approach that links the detection of curved reflection symmetries to produce symmetry-constrained segments of structures regions in real images with clutter. For curved reflection symmetry detection, we leverage on patch-based symmetric features to train a Structured Random Forest classifier that detects multiscaled curved symmetries in 2D images. Next, using these curved symmetries, we modulate a novel symmetry-constrained foreground-background segmentation by their symmetry scores so that we enforce global symmetrical consistency in the final segmentation. This is achieved by imposing a pairwise symmetry prior that encourages symmetric pixels to have the same labels over a MRF-based representation of the input image edges, and the final segmentation is obtained via graph-cuts. Experimental results over four publicly available datasets containing annotated symmetric structures: 1) SYMMAX-300 [38], 2) BSD-Parts, 3) Weizmann Horse (both from [18]) and 4) NY-roads [35] demonstrate the approach's applicability to different environments with state-of-the-art performance. The main challenge in Super Resolution (SR) is to discover the mapping between the low-and high-resolution manifolds of image patches, a complex ill-posed problem which has recently been addressed through piecewise linear regression with promising results. In this paper we present a novel regression-based SR algorithm that benefits from an extended knowledge of the structure of both manifolds. We propose a transform that collapses the 16 variations induced from the dihedral group of transforms (i.e. rotations, vertical and horizontal reflections) and antipodality (i.e. diametrically opposed points in the unitary sphere) into a single primitive. The key idea of our transform is to study the different dihedral elements as a group of symmetries within the high-dimensional manifold. We obtain the respective set of mirror-symmetry axes by means of a frequency analysis of the dihedral elements, and we use them to collapse the redundant variability through a modified symmetry distance. The experimental validation of our algorithm shows the effectiveness of our approach, which obtains competitive quality with a dictionary of as little as 32 atoms (reducing other methods' dictionaries by at least a factor of 32) and further pushing the state-of-the-art with a 1024 atoms dictionary. We generalize the concept of bilateral reflection symmetry to curved glide-reflection symmetry in 2D euclidean space, such that classic reflection symmetry becomes one of its six special cases. We propose a local feature-based approach for curved glide-reflection symmetry detection from real, unsegmented 2D images. Furthermore, we apply curved glide-reflection axis detection for curved reflection surface detection in 3D images. Our method discovers, groups, and connects statistically dominant local glide-reflection axes in an Axis-Parameter-Space (APS) without preassumptions on the types of reflection symmetries. Quantitative evaluations and comparisons against state-of-the-art algorithms on a diverse 64-test-image set and 1,125 Swedish leaf-data images show a promising average detection rate of the proposed algorithm at 80 and 40 percent, respectively, and superior performance over existing reflection symmetry detection algorithms. Potential applications in computer vision, particularly biomedical imaging, include saliency detection from unsegmented images and quantification of deviations from normality. We make our 64-test-image set publicly available. A simple and fast reflectional symmetry detection algorithm has been developed in this paper. The algorithm employs only the original gray scale image and the gradient information of the image, and it is able to detect multiple reflectional symmetry axes of an object in the image. The directions of the symmetry axes are obtained from the gradient orientation histogram of the input gray scale image by using the Fourier method. Both synthetic and real images have been tested using the proposed algorithm. We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution layers. G-convolutions increase the expressive capacity of the network without increasing the number of parameters. Group convolution layers are easy to use and can be implemented with negligible computational overhead for discrete groups generated by translations, reflections and rotations. G-CNNs achieve state of the art results on CIFAR10 and rotated MNIST. We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution layers. G-convolutions increase the expressive capacity of the network without increasing the number of parameters. Group convolution layers are easy to use and can be implemented with negligible computational overhead for discrete groups generated by translations, reflections and rotations. G-CNNs achieve state of the art results on CI- FAR10 and rotated MNIST. Many classes of images exhibit rotational symmetry. Convolutional neural networks are sometimes trained using data augmentation to exploit this, but they are still required to learn the rotation equivariance properties from the data. Encoding these properties into the network architecture, as we are already used to doing for translation equivariance by using convolutional layers, could result in a more efficient use of the parameter budget by relieving the model from learning them. We introduce four operations which can be inserted into neural network models as layers, and which can be combined to make these models partially equivariant to rotations. They also enable parameter sharing across different orientations. We evaluate the effect of these architectural modifications on three datasets which exhibit rotational symmetry and demonstrate improved performance with smaller models. We present an algorithm for detecting multiple rotational symmetries in natural images. Given an image, its gradient magnitude field is computed, and information from the gradients is spread using a diffusion process in the form of a gradient vector flow (GVF) field. We construct a graph whose nodes correspond to pixels in tire image, connecting points that are likely to be rotated versions of one another The n-cycles present in tire graph are made to vote for C sub n symmetries, their votes being weighted by the errors in transformation between GVF in the neighborhood of the voting points, and the irregularity of the n-sided polygons formed by the voters. The votes are accumulated at tire centroids of possible rotational symmetries, generating a confidence map for each order of symmetry. We tested the method with several natural images. A method for efficient detection of the dominant local reflectional symmetry in grey level images is described. The general approach is to define a local measure of reflectional symmetry that transforms the symmetry detection problem to an optimization problem, and obtain the symmetric regions by an efficient global optimization algorithm. The symmetry of a 1D function can be measured in the frequency domain as the fraction of its energy that resides in symmetric Fourier basis functions. This approach is extended to two dimensions. Locality can be formally treated in terms of the Gabor decomposition and implemented via soft windowing. The resulting measure is a complicated multimodal function of the location of the center of the supporting region, its size, and the orientation of the symmetry axis. A new probabilistic generic algorithm is applied to the determination of the global maximum of the reflectional symmetry function. Less than one thousand evaluations of the local symmetry measure are typically needed in order to locate the dominant symmetry in natural, wildlife test images. Object detection in very high resolution optical remote sensing images is a fundamental problem faced for remote sensing image analysis. Due to the advances of powerful feature representations, machine-learning-based object detection is receiving increasing attention. Although numerous feature representations exist, most of them are handcrafted or shallow-learning-based features. As the object detection task becomes more challenging, their description capability becomes limited or even impoverished. More recently, deep learning algorithms, especially convolutional neural networks (CNNs), have shown their much stronger feature representation power in computer vision. Despite the progress made in nature scene images, it is problematic to directly use the CNN feature for object detection in optical remote sensing images because it is difficult to effectively deal with the problem of object rotation variations. To address this problem, this paper proposes a novel and effective approach to learn a rotation-invariant CNN (RICNN) model for advancing the performance of object detection, which is achieved by introducing and learning a new rotation-invariant layer on the basis of the existing CNN architectures. However, different from the training of traditional CNN models that only optimizes the multinomial logistic regression objective, our RICNN model is trained by optimizing a new objective function via imposing a regularization constraint, which explicitly enforces the feature representations of the training samples before and after rotating to be mapped close to each other, hence achieving rotation invariance. To facilitate training, we first train the rotation-invariant layer and then domain-specifically fine-tune the whole RICNN network to further boost the performance. Comprehensive evaluations on a publicly available ten-class object detection data set demonstrate the effectiveness of the proposed method. Symmetry in visual data represents repeated patterns or shapes that is easily found in natural and human-made objects. Symmetry pattern on an object works as a salient visual feature attracting human attention and letting the object to be easily recognized. Most existing symmetry detection methods are based on sparsely detected local features describing the appearance of their neighborhood, which have difficulty in capturing object structure mostly supported by edges and contours. In this work, we propose a new reflection symmetry detection method extracting robust 4-dimensional Appearance of Structure descriptors based on a set of outstanding neighbourhood edge segments in multiple scales. Our experimental evaluations on multiple public symmetry detection datasets show promising reflection symmetry detection results on challenging real world and synthetic images. We investigate the constraints placed on the image projection of a planar object having local reflectional symmetry. Under the affine approximation to projection, we demonstrate an efficient (low-complexity) algorithm for detecting and verifying symmetries despite the distorting effects of image skewing. The symmetries are utilized for three distinct tasks: first, determining image back-projection up to a similarity transformation ambiguity; second, determining the object plane orientation (slant and tilt); and third, as a test for non-coplanarity amongst a collection of objects. These results are illustrated throughout with examples from images of real scenes.