Color and intensity are two important components in an image. Usually, groups of image pixels, which are similar in color or intensity, are an informative representation for an object. They are therefore particularly suitable for computer vision tasks, such as saliency detection and object proposal generation. However, image pixels, which share a similar real-world color, may be quite different since colors are often distorted by intensity. In this paper, we reinvestigate the affinity matrices originally used in image segmentation methods based on spectral clustering. A new affinity matrix, which is robust to color distortions, is formulated for object discovery. Moreover, a cohesion measurement (CM) for object regions is also derived based on the formulated affinity matrix. Based on the new CM, a novel object discovery method is proposed to discover objects latent in an image by utilizing the eigenvectors of the affinity matrix. Then we apply the proposed method to both saliency detection and object proposal generation. Experimental results on several evaluation benchmarks demonstrate that the proposed CM-based method has achieved promising performance for these two tasks.
An affinity matrix plays an important role in the spectral graph theory @cite_14 . For example, @cite_38 propose to solve the perceptual grouping problem using normalized cuts. They set up a weighted graph and set the weight of each edge connecting two nodes to be a measure of the affinity between the two nodes. Then image segmentation is treated as a graph partitioning problem, which is to find the minimum cut of a graph by solving a generalized eigenvalue problem on the constructed Laplacian matrix. The second-to-last eigenvectors turn out to be indicator vectors for partitioning the graph. @cite_30 consider an asymmetric variant of the cost function in @cite_38 , and define one of the two subsets of a graph to be the foreground and define its complement to be the background. Based on the modified normalized cuts, they derive a foreground cut method by performing affinity factorization on the affinity matrix. The first eigenvector of the affinity matrix with the largest eigenvalue contains salient objects.
@cite_30 @cite_38 @cite_14
2976981929 2963630186 2160915541 1640070940
The foreground group in a scene may be 'discovered' and computed as a factorized approximation to the pairwise affinity of the elements in the scene. A pointwise approximation of the pairwise affinity information may in fact be interpreted as a 'saliency' index, and the foreground of the scene may be obtained by thresholding it. An algorithm called 'affinity factorization' is thus obtained which may be used for grouping. The affinity factorization algorithm is demonstrated on displays composed of points, of lines and of brightness values. Its relationship to the Shi-Malik normalized cuts algorithms is explored both analytically and experimentally. The affinity factorization algorithm is shown to be computationally efficient (O(n) floating-point operations for a scene composed of n elements) and to perform well on displays where the background is unstructured. Generalizations to solve more complex problems are also discussed. Spectral embedding provides a framework for solving perceptual organization problems, including image segmentation and figure ground organization. From an affinity matrix describing pairwise relationships between pixels, it clusters pixels into regions, and, using a complex-valued extension, orders pixels according to layer. We train a convolutional neural network (CNN) to directly predict the pair-wise relationships that define this affinity matrix. Spectral embedding then resolves these predictions into a globally-consistent segmentation and figure ground organization of the scene. Experiments demonstrate significant benefit to this direct coupling compared to prior works which use explicit intermediate stages, such as edge detection, on the pathway from image to affinities. Our results suggest spectral embedding as a powerful alternative to the conditional random field (CRF)-based globalization schemes typically coupled to deep neural networks. This paper studies the subspace segmentation problem. Given a set of data points drawn from a union of subspaces, the goal is to partition them into their underlying subspaces they were drawn from. The spectral clustering method is used as the framework. It requires to find an affinity matrix which is close to block diagonal, with nonzero entries corresponding to the data point pairs from the same subspace. In this work, we argue that both sparsity and the grouping effect are important for subspace segmentation. A sparse affinity matrix tends to be block diagonal, with less connections between data points from different subspaces. The grouping effect ensures that the highly corrected data which are usually from the same subspace can be grouped together. Sparse Subspace Clustering (SSC), by using l1-minimization, encourages sparsity for data selection, but it lacks of the grouping effect. On the contrary, Low-Rank Representation (LRR), by rank minimization, and Least Squares Regression (LSR), by l2-regularization, exhibit strong grouping effect, but they are short in subset selection. Thus the obtained affinity matrix is usually very sparse by SSC, yet very dense by LRR and LSR. In this work, we propose the Correlation Adaptive Subspace Segmentation (CASS) method by using trace Lasso. CASS is a data correlation dependent method which simultaneously performs automatic data selection and groups correlated data together. It can be regarded as a method which adaptively balances SSC and LSR. Both theoretical and experimental results show the effectiveness of CASS. The foreground group in a scene may be ‘discovered’ and computed as a factorized approximation to the pairwise affinity of the elements in the scene. A pointwise approximation of the pairwise affinity information may in fact be interpreted as a ‘saliency’ index, and the foreground of the scene may be obtained by thresholding it. An algorithm called ‘affinity factorization’ is thus obtained which may be used for grouping.