An end-to-end trainable ConvNet architecture, that learns to harness the power of shape representation for matching disparate image pairs, is proposed. Disparate image pairs are deemed those that exhibit strong affine variations in scale, viewpoint and projection parameters accompanied by the presence of partial or complete occlusion of objects and extreme variations in ambient illumination. Under these challenging conditions, neither local nor global feature-based image matching methods, when used in isolation, have been observed to be effective. The proposed correspondence determination scheme for matching disparate images exploits high-level shape cues that are derived from low-level local feature descriptors, thus combining the best of both worlds. A graph-based representation for the disparate image pair is generated by constructing an affinity matrix that embeds the distances between feature points in two images, thus modeling the correspondence determination problem as one of graph matching. The eigenspectrum of the affinity matrix, i.e., the learned global shape representation, is then used to further regress the transformation or homography that defines the correspondence between the source image and target image. The proposed scheme is shown to yield state-of-the-art results for both, coarse-level shape matching as well as fine point-wise correspondence determination.
Both, local point-based and global shape-based image matching approaches have their advantages and shortcomings. While global shape-based descriptors are well behaved and less sensitive to outliers, they, in isolation, are insufficient to compute point-wise correspondences since they do not explicitly encode keypoint information as appearance-based descriptors do @cite_8 @cite_26 . Although global shape descriptors are shown to perform well when the image pairs exhibit significant shape deformations due to changes in viewpoint, their performance suffers in the presence of strong shape articulations @cite_26 . Region-based global descriptors are also vulnerable to instances of partial occlusions. While global shape representations have the advantage of being able to employ global shape cues for matching, their computational complexity and inability to compute accurate point-wise correspondences makes them unsuitable for most practical applications that demand reliable point-wise correspondences. In contrast, local point-based descriptors, in theory, are capable of yielding more reliable keypoint correspondences, but are often plagued by noisy matches and can also prove to be computationally expensive.
@cite_26 @cite_8
2004134605 1980807298 2753872511 2108444897
Motivated by the discriminative ability of shape information and local patterns in object recognition, this paper proposes a window-based object descriptor that integrates both cues. In particular, contour templates representing object shape are used to derive a set of so-called key points at which local appearance features are extracted. These key points are located using an improved template matching method that utilises both spatial and orientation information in a simple and effective way. At each of the extracted key points, a new local appearance feature, namely non-redundant local binary pattern (NR-LBP), is computed. An object descriptor is formed by concatenating the NR-LBP features from all key points to encode the shape as well as the appearance of the object. The proposed descriptor was extensively tested in the task of detecting humans from static images on the commonly used MIT and INRIA datasets. The experimental results have shown that the proposed descriptor can effectively describe non-rigid objects with high articulation and improve the detection rate compared to other state-of-the-art object descriptors. We suggest a novel shape contour descriptor for shape matching and retrieval. The new descriptor is called contour points distribution histogram (CPDH) which is based on the distribution of points on object contour under polar coordinates. CPDH not only conforms to the human visual perception but also the computational complexity of it is low. Invariant to scale and translation are the intrinsic properties of CPDH and the problem of the invariant to rotation can be partially resolved in the matching process. After the CPDHs of images are generated, the similarity value of the images is obtained by EMD (Earth Mover's Distance) metric. In order to make the EMD method used effectively for the matching of CPDHs, we also develop a new approach to the ground distance used in the EMD metric under polar coordinates. Experimental results of image retrieval demonstrate that the novel descriptor has a strong capability in handling a variety of shapes. We present a new local descriptor for 3D shapes, directly applicable to a wide range of shape analysis problems such as point correspondences, semantic segmentation, affordance prediction, and shape-to-scan matching. The descriptor is produced by a convolutional network that is trained to embed geometrically and semantically similar points close to one another in descriptor space. The network processes surface neighborhoods around points on a shape that are captured at multiple scales by a succession of progressively zoomed-out views, taken from carefully selected camera positions. We leverage two extremely large sources of data to train our network. First, since our network processes rendered views in the form of 2D images, we repurpose architectures pretrained on massive image datasets. Second, we automatically generate a synthetic dense point correspondence dataset by nonrigid alignment of corresponding shape parts in a large collection of segmented 3D models. As a result of these design choices, our network effectively encodes multiscale local context and fine-grained surface detail. Our network can be trained to produce either category-specific descriptors or more generic descriptors by learning from multiple shape categories. Once trained, at test time, the network extracts local descriptors for shapes without requiring any part segmentation as input. Our method can produce effective local descriptors even for shapes whose category is unknown or different from the ones used while training. We demonstrate through several experiments that our learned local descriptors are more discriminative compared to state-of-the-art alternatives and are effective in a variety of shape analysis applications. We develop an approach to object recognition based on matching shapes and using a resulting measure of similarity in a nearest neighbor classifier. The key algorithmic problem here is that of finding pointwise correspondences between an image shape and a stored prototype shape. We introduce a new shape descriptor, the shape context, which makes this possible, using a simple and robust algorithm. The shape context at a point captures the distribution over relative positions of other shape points and thus summarizes global shape in a rich, local descriptor. We demonstrate that shape contexts greatly simplify recovery of correspondences between points of two given shapes. Once shapes are aligned, shape contexts are used to define a robust score for measuring shape similarity. We have used this score in a nearest-neighbor classifier for recognition of hand written digits as well as 3D objects, using exactly the same distance function. On the benchmark MNIST dataset of handwritten digits, this yields an error rate of 0.63 , outperforming other published techniques.