Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism into a computational model remains elusive. Encouraged by a new discovery (CVPR 2016) demonstrating extremely high inter-person accuracy of human perceived symmetries in the wild, we have created the first deep-learning neural network for reflection and rotation symmetry detection (Sym-NET), trained on photos from MS-COCO (Common Object in COntext) dataset with nearly 11K symmetry-labels from more than 400 human observers. We employ novel methods to convert discrete human labels into symmetry heatmaps, capture symmetry densely in an image and quantitatively evaluate Sym-NET against multiple existing computer vision algorithms. Using the symmetry competition testsets from CVPR 2013 and unseen MS-COCO photos, Sym-NET comes out as the winner with significantly superior performance over all other competitors. Beyond mathematically well-defined symmetries on a plane, Sym-NET demonstrates abilities to identify viewpoint-varied 3D symmetries, partially occluded symmetrical objects and symmetries at a semantic level.
Reflection symmetry algorithms fall into two different categories depending on whether they detect sparse symmetries (straight lines or curves) @cite_36 @cite_76 @cite_2 @cite_72 @cite_51 @cite_17 or a dense heatmap @cite_41 @cite_73 @cite_9 . The most common sparse approach to detect reflection symmetry is to match up symmetric points or contours in the image to determine midpoint and direction of the symmetry axis @cite_53 @cite_60 @cite_51 @cite_56 @cite_19 @cite_42 . These approaches often use a Hough transform to accumulate the axes of reflection, derived from the matched feature's midpoints and angles, and vote on the dominant symmetries. Atadjanov and Lee @cite_57 extend the Loy and Eklundh @cite_51 algorithm by taking the matched keypoints and then comparing the histogram of curvature around the keypoints. Wang al @cite_42 uses local affine invariant edge correspondences to make the algorithms more resilient to perspective distortion contours. The method does not use a Hough space to vote, opting instead to use an affine invariant pairwise (dis)similarity metric to vote for symmetries.
@cite_73 @cite_60 @cite_36 @cite_41 @cite_9 @cite_53 @cite_42 @cite_56 @cite_57 @cite_19 @cite_72 @cite_2 @cite_76 @cite_51 @cite_17
2003233837 2276958818 2160306297 2217188300 2060321377 1965837944 1541124573 1973016074 2154063579 174734558 2045323633 1554810293 2952438563 2591692311 2116015281
Reflection symmetry detection receives increasing attentions in recent years. The state-of-the-art algorithms mainly use the matching of intensity-based features (such as the SIFT) within a single image to find symmetry axes. This paper proposes a novel approach by establishing the correspondence of locally affine invariant edge-based features, which are superior to the intensity based in the aspects that it is insensitive to illumination variations, and applicable to textureless objects. The locally affine invariance is achieved by simple linear algebra for efficient and robust computations, making the algorithm suitable for detections under object distortions like perspective projection. Commonly used edge detectors and a voting process are, respectively, used before and after the edge description and matching steps to form a complete reflection detection pipeline. Experiments are performed using synthetic and real-world images with both multiple and single reflection symmetry axis. The test results are compared with existing algorithms to validate the proposed method. An unsupervised procedure to local symmetry detection in natural images is proposed.Achieved via a Hough-style voting approach made at different scales and coordinate spaces.A final denoising scheme aims at removing noise and spurious local symmetries.Experiments have been conducted on the recent extension of the Berkeley Segmentation Dataset.The proposed method performs well compared to the state-of-the-art algorithms. Display Omitted In this paper, we propose a new unsupervised and simple approach to local symmetry detection of ribbon-like structure in natural images. The proposed model consists in quantifying the presence of a partial medial axis segment, existing between each pair of (preliminary detected) line segments delineating the boundary of two textured regions, by a set of heuristics related both to the geometrical structure of each pair of line segments and its ability to locally delimit a homogeneous texture region in the image. This semi-local approach is finally embedded in a two-step algorithm with an amplification step, via a Hough-style voting approach achieved at different scales and coordinate spaces which aims at determining the dominant local symmetries present in the image and a final denoising step, via an averaging procedure, which aims at removing noise and spurious local symmetries. The experiments, reported in this paper and conducted on the recent extension of the Berkeley Segmentation Dataset for the local symmetry detection task, demonstrate that the proposed symmetry detector performs well compared to the best existing state-of-the-art algorithms recently proposed in the literature. Local reflection symmetry detection in nature images is a quite important but challenging task in computer vision. The main obstacle is both the scales and the orientations of symmetric structure are unknown. The multiple instance learning (MIL) framework sheds lights onto this task owing to its capability to well accommodate the unknown scales and orientations of the symmetric structures. However, to differentiate symmetry vs non-symmetry remains to face extreme confusions caused by clutters scenes and ambiguous object structures. In this paper, we propose a novel multiple instance learning framework for local reflection symmetry detection, named multiple instance subspace learning (MISL), which instead learns a group of models respectively on well partitioned subspaces. To obtain such subspaces, we propose an efficient dividing strategy under MIL setting, named partial random projection tree (PRPT), by taking advantage of the fact that each sample (bag) is represented by the proposed symmetry features computed at specific scale and orientation combinations (instances). Encouraging experimental results on two datasets demonstrate that the proposed local reflection symmetry detection method outperforms current state-of-the-arts. HighlightsWe perform clustering on samples represented by multiple instances.We learn a group of MIL classifiers on subspaces.We report state-of-the-arts results on the symmetry detection benchmark. Symmetry, as one of the key components of Gestalt theory, provides an important mid-level cue that serves as input to higher visual processes such as segmentation. In this work, we propose a complete approach that links the detection of curved reflection symmetries to produce symmetry-constrained segments of structures regions in real images with clutter. For curved reflection symmetry detection, we leverage on patch-based symmetric features to train a Structured Random Forest classifier that detects multiscaled curved symmetries in 2D images. Next, using these curved symmetries, we modulate a novel symmetry-constrained foreground-background segmentation by their symmetry scores so that we enforce global symmetrical consistency in the final segmentation. This is achieved by imposing a pairwise symmetry prior that encourages symmetric pixels to have the same labels over a MRF-based representation of the input image edges, and the final segmentation is obtained via graph-cuts. Experimental results over four publicly available datasets containing annotated symmetric structures: 1) SYMMAX-300 [38], 2) BSD-Parts, 3) Weizmann Horse (both from [18]) and 4) NY-roads [35] demonstrate the approach's applicability to different environments with state-of-the-art performance. A method for efficient detection of the dominant local reflectional symmetry in grey level images is described. The general approach is to define a local measure of reflectional symmetry that transforms the symmetry detection problem to an optimization problem, and obtain the symmetric regions by an efficient global optimization algorithm. The symmetry of a 1D function can be measured in the frequency domain as the fraction of its energy that resides in symmetric Fourier basis functions. This approach is extended to two dimensions. Locality can be formally treated in terms of the Gabor decomposition and implemented via soft windowing. The resulting measure is a complicated multimodal function of the location of the center of the supporting region, its size, and the orientation of the symmetry axis. A new probabilistic generic algorithm is applied to the determination of the global maximum of the reflectional symmetry function. Less than one thousand evaluations of the local symmetry measure are typically needed in order to locate the dominant symmetry in natural, wildlife test images. We propose a novel approach for detecting partial reflectional symmetry in images. Our method consists of two principal stages: candidate selection and validation. In the first step, candidates for mirror-symmetric patches are identified using an existing heuristic procedure based on Hough voting. The candidates are then validated using a principled statistical procedure inspired from the a contrario theory, which minimizes the number of false positives. Our algorithm uses integral image properties to enhance the execution time. We present two methods for detecting symmetry in images, one based directly on the intensity values and another one based on a discrete representation of local orientation. A symmetry finder has been developed which uses the intensity-based method to search an image for compact regions which display some degree of mirror symmetry due to intensity similarities across a straight axis. In a different approach, we look at symmetry as a bilateral relationship between local orientations. A symmetryenhancing edge detector is presented which indicates edges dependent on the orientations at two different image positions. SEED, as we call it, is a detector element implemented by a feedforward network that holds the symmetry conditions. We use SEED to find the contours of symmetric objects of which we know the axis of symmetry from the intensity-based symmetry finder. The methods presented have been applied to the problem of visually guided car-following. Real-time experiments with a system for automatic headway control on motorways have been successful. We investigate the constraints placed on the image projection of a planar object having local reflectional symmetry. Under the affine approximation to projection, we demonstrate an efficient (low-complexity) algorithm for detecting and verifying symmetries despite the distorting effects of image skewing. The symmetries are utilized for three distinct tasks: first, determining image back-projection up to a similarity transformation ambiguity; second, determining the object plane orientation (slant and tilt); and third, as a test for non-coplanarity amongst a collection of objects. These results are illustrated throughout with examples from images of real scenes. This paper addresses the problem of detecting axes of bilateral symmetry in images. In order to achieve robustness to variation in illumination, only edge-gradient information is used. To overcome the problem of edge breaks, a potential field is developed from the edge map which spreads the information in the image plane. Pairs of points in the image plane are made to vote for their axes of symmetry with some confidence values. To make the method robust to overlapping objects, only local features in the form of Taylor coefficients are used for quantifying symmetry. We define an axis of symmetry histogram, which is used to accumulate the weighted votes for all possible axes of symmetry. To reduce the computational complexity of voting, a hashing scheme is proposed, wherein pairs of points, whose potential fields are too asymmetric, are pruned by not being counted for the vote. Experimental results indicate that the proposed method is fairly robust to edge breaks and is able to detect symmetries even when only 0.05 of the possible pairs are used for voting. In this work we propose a learning-based approach to symmetry detection in natural images. We focus on ribbon-like structures, i.e. contours marking local and approximate reflection symmetry and make three contributions to improve their detection. First, we create and make publicly available a ground-truth dataset for this task by building on the Berkeley Segmentation Dataset. Second, we extract features representing multiple complementary cues, such as grayscale structure, color, texture, and spectral clustering information. Third, we use supervised learning to learn how to combine these cues, and employ MIL to accommodate the unknown scale and orientation of the symmetric structures. We systematically evaluate the performance contribution of each individual component in our pipeline, and demonstrate that overall we consistently improve upon results obtained using existing alternatives. We present an algorithm for multi-scale partial intrinsic symmetry detection over 2D and 3D shapes, where the scale of a symmetric region is defined by intrinsic distances between symmetric points over the region. To identify prominent symmetric regions which overlap and vary in form and scale, we decouple scale extraction and symmetry extraction by performing two levels of clustering. First, significant symmetry scales are identified by clustering sample point pairs from an input shape. Since different point pairs can share a common point, shape regions covered by points in different scale clusters can overlap. We introduce the symmetry scale matrix (SSM), where each entry estimates the likelihood two point pairs belong to symmetries at the same scale. The pair-to-pair symmetry affinity is computed based on a pair signature which encodes scales. We perform spectral clustering using the SSM to obtain the scale clusters. Then for all points belonging to the same scale cluster, we perform the second-level spectral clustering, based on a novel point-to-point symmetry affinity measure, to extract partial symmetries at that scale. We demonstrate our algorithm on complex shapes possessing rich symmetries at multiple scales. We propose a novel, self-validating approach for detecting curved reflection symmetry patterns from real, unsegmented images. Our method benefits from the observation that any curved symmetry pattern can be approximated by a sequence of piecewise rigid reflection patterns. Pairs of symmetric feature points are first detected (including both inliers and outliers) and treated as 'particles'. Multiple-hypothesis sampling and pruning are used to sample a smooth path going through inlier particles to recover the curved reflection axis. Our approach generates an explicit supporting region of the curved reflection symmetry, which is further used for intermediate self-validation, making the detection process more robust than prior state-of-the-art algorithms. Experimental results on 200+ images demonstrate the effectiveness and superiority of the proposed approach. In this paper, we establish a baseline for object symmetry detection in complex backgrounds by presenting a new benchmark and an end-to-end deep learning approach, opening up a promising direction for symmetry detection in the wild. The new benchmark, named Sym-PASCAL, spans challenges including object diversity, multi-objects, part-invisibility, and various complex backgrounds that are far beyond those in existing datasets. The proposed symmetry detection approach, named Side-output Residual Network (SRN), leverages output Residual Units (RUs) to fit the errors between the object symmetry groundtruth and the outputs of RUs. By stacking RUs in a deep-to-shallow manner, SRN exploits the 'flow' of errors among multiple scales to ease the problems of fitting complex outputs with limited layers, suppressing the complex backgrounds, and effectively matching object symmetry of different scales. Experimental results validate both the benchmark and its challenging aspects related to realworld images, and the state-of-the-art performance of our symmetry detection approach. The benchmark and the code for SRN are publicly available at this https URL. In this paper, we establish a baseline for object symmetry detection in complex backgrounds by presenting a new benchmark and an end-to-end deep learning approach, opening up a promising direction for symmetry detection in the wild. The new benchmark, named Sym-PASCAL, spans challenges including object diversity, multi-objects, part-invisibility, and various complex backgrounds that are far beyond those in existing datasets. The proposed symmetry detection approach, named Side-output Residual Network (SRN), leverages output Residual Units (RUs) to fit the errors between the object symmetry ground-truth and the outputs of RUs. By stacking RUs in a deep-to-shallow manner, SRN exploits the flow of errors among multiple scales to ease the problems of fitting complex outputs with limited layers, suppressing the complex backgrounds, and effectively matching object symmetry of different scales. Experimental results validate both the benchmark and its challenging aspects related to real-world images, and the state-of-the-art performance of our symmetry detection approach. The benchmark and the code for SRN are publicly available at https: github.com KevinKecc SRN. We generalize the concept of bilateral reflection symmetry to curved glide-reflection symmetry in 2D euclidean space, such that classic reflection symmetry becomes one of its six special cases. We propose a local feature-based approach for curved glide-reflection symmetry detection from real, unsegmented 2D images. Furthermore, we apply curved glide-reflection axis detection for curved reflection surface detection in 3D images. Our method discovers, groups, and connects statistically dominant local glide-reflection axes in an Axis-Parameter-Space (APS) without preassumptions on the types of reflection symmetries. Quantitative evaluations and comparisons against state-of-the-art algorithms on a diverse 64-test-image set and 1,125 Swedish leaf-data images show a promising average detection rate of the proposed algorithm at 80 and 40 percent, respectively, and superior performance over existing reflection symmetry detection algorithms. Potential applications in computer vision, particularly biomedical imaging, include saliency detection from unsegmented images and quantification of deviations from normality. We make our 64-test-image set publicly available.