Image retrieval based on deep convolutional features has demonstrated state-of-the-art performance in popular benchmarks. In this paper, we present a unified solution to address deep convolutional feature aggregation and image re-ranking by simulating the dynamics of heat diffusion. A distinctive problem in image retrieval is that repetitive or features tend to dominate final image representations, resulting in representations less distinguishable. We show that by considering each deep feature as a heat source, our unsupervised aggregation method is able to avoid over-representation of features. We additionally provide a practical solution for the proposed aggregation method and further show the efficiency of our method in experimental evaluation. Inspired by the aforementioned deep feature aggregation method, we also propose a method to re-rank a number of top ranked images for a given query image by considering the query as the heat source. Finally, we extensively evaluate the proposed approach with pre-trained and fine-tuned deep networks on common public benchmarks and show superior performance compared to previous work.
Anisotropic diffusion has been applied to many computer vision problems, such as image segmentation @cite_55 @cite_43 , saliency detection @cite_35 @cite_20 , and clustering @cite_46 @cite_15 . In these applications, diffusion is used for finding central points by capturing the intrinsic manifold structure of the data. Our deep feature aggregation method models the problem in the opposite direction, and we differentiate them by weakening deep features that are densely connected to other features with high similarities.
@cite_35 @cite_15 @cite_55 @cite_43 @cite_46 @cite_20
1985984012 2125436763 1970899458 1984918059
In diffusion-based saliency detection, an image is partitioned into superpixels and mapped to a graph, with superpixels as nodes and edge strengths proportional to superpixel similarity. Saliency information is then propagated over the graph using a diffusion process, whose equilibrium state yields the object saliency map. The optimal solution is the product of a propagation matrix and a saliency seed vector that contains a prior saliency assessment. This is obtained from either a bottom-up saliency detector or some heuristics. In this work, we propose a method to learn optimal seeds for object saliency. Two types of features are computed per superpixel: the bottom-up saliency of the superpixel region and a set of mid-level vision features informative of how likely the superpixel is to belong to an object. The combination of features that best discriminates between object and background saliency is then learned, using a large-margin formulation of the discriminant saliency principle. The propagation of the resulting saliency seeds, using a diffusion process, is finally shown to outperform the state of the art on a number of salient object detection datasets. Seeded image segmentation is a popular type of supervised image segmentation in computer vision and image processing. Previous methods of seeded image segmentation treat the image as a weighted graph and minimize an energy function on the graph to produce a segmentation. In this paper, we propose to conduct the seeded image segmentation according to the result of a heat diffusion process in which the seeded pixels are considered to be the heat sources and the heat diffuses on the image starting from the sources. After the diffusion reaches a stable state, the image is segmented based on the pixel temperatures. It is also shown that our proposed framework includes the RandomWalk algorithm for image segmentation as a special case which diffuses only along the two coordinate axes. To better control diffusion, we propose to incorporate the attributes (such as the geometric structure) of the image into the diffusion process, yielding an anisotropic diffusion method for image segmentation. The experiments show that the proposed anisotropic diffusion method usually produces better segmentation results. In particular, when the method is tested using the groundtruth dataset of Microsoft Research Cambridge (MSRC), an error rate of 4.42 can be achieved, which is lower than the reported error rates of other state-of-the-art algorithms. The saliency of regions or objects in an image can be significantly boosted if they recur in multiple images. Leveraging this idea, cosegmentation jointly segments common regions from multiple images. In this paper, we propose CoSand, a distributed cosegmentation approach for a highly variable large-scale image collection. The segmentation task is modeled by temperature maximization on anisotropic heat diffusion, of which the temperature maximization with finite K heat sources corresponds to a K-way segmentation that maximizes the segmentation confidence of every pixel in an image. We show that our method takes advantage of a strong theoretic property in that the temperature under linear anisotropic diffusion is a submodular function; therefore, a greedy algorithm guarantees at least a constant factor approximation to the optimal solution for temperature maximization. Our theoretic result is successfully applied to scalable cosegmentation as well as diversity ranking and single-image segmentation. We evaluate CoSand on MSRC and ImageNet datasets, and show its competence both in competitive performance over previous work, and in much superior scalability. In this paper we introduce an efficient, effective and scalable clustering method denoted as Replicator Graph Clustering. Our method takes measures of similarity between pairs of data points (i. e. an affinity matrix) as input and identifies a set of clusters and unique cluster assignments in a fully unsupervised manner, where the cluster granularity is adaptable by a single parameter. We provide clustering results in three subsequent steps: (a) diffusing affinities by finding personalized evolutionary stable strategies of non-cooperative games (b) building a mutual k-nearest neighbor graph representing the underlying manifold and (c) applying a graph based clustering strategy which identifies the final clusters. Individual steps have low computational complexity which leads to an efficient clustering method, scaling well with an increasing number of data points. Experimental evaluation demonstrates competitive performance to state-of-the-art in several application fields.