Multi-index fusion has demonstrated impressive performances in retrieval task by integrating different visual representations in a unified framework. However, previous works mainly consider propagating similarities via neighbor structure, ignoring the high order information among different visual representations. In this paper, we propose a new multi-index fusion scheme for image retrieval. By formulating this procedure as a multilinear based optimization problem, the complementary information hidden in different indexes can be explored more thoroughly. Specially, we first build our multiple indexes from various visual representations. Then a so-called index-specific functional matrix, which aims to propagate similarities, is introduced for updating the original index. The functional matrices are then optimized in a unified tensor space to achieve a refinement, such that the relevant images can be pushed more closer. The optimization problem can be efficiently solved by the augmented Lagrangian method with theoretical convergence guarantee. Unlike the traditional multi-index fusion scheme, our approach embeds the multi-index subspace structure into the new indexes with sparse constraint, thus it has little additional memory consumption in online query stage. Experimental evaluation on three benchmark datasets reveals that the proposed approach achieves the state-of-the-art performance, i.e., N-score 3.94 on UKBench, mAP 94.1 on Holiday and 62.39 on Market-1501.
To overcome these drawbacks, some works focus on fusing visual features on index level. A common assumption shared in these methods is that: two images, which are nearest neighbors to each other under one type of visual representation, are probably to be true related. By pushing them closer in other visual feature spaces, the search accuracy can be greatly promoted. Under the guidance of this principle, the proposed collaborative index embedding method @cite_6 , which is most relevant to our work, utilize an alternating index update scheme to fuse feature. By enriching the corresponding feature, it refine the neighborhood structures to improve the retrieval accuracy. Chen @cite_32 extend this model for the multi-index fusion problem. However, both of these methods neglect the distance information of original feature space. More importantly, high order information is more or less ignored.
@cite_32 @cite_6
2741640202 2591614014
Different kinds of features hold some distinct merits, making them complementary to each other. Inspired by this idea an index level multiple feature fusion scheme via similarity matrix pooling is proposed in this paper. We first compute the similarity matrix of each index, and then a novel scheme is used to pool on these similarity matrices for updating the original indices. Compared with the existing fusion schemes, the proposed scheme performs feature fusion at index level to save memory and reduce computational complexity. On the other hand, the proposed scheme treats different kinds of features adaptively based on its importance, thus improves retrieval accuracy. The performance of the proposed approach is evaluated using two public datasets, which significantly outperforms the baseline methods in retrieval accuracy with low memory consumption and computational complexity. In content-based image retrieval, SIFT feature and the feature from deep convolutional neural network (CNN) have demonstrated promising performance. To fully explore both visual features in a unified framework for effective and efficient retrieval, we propose a collaborative index embedding method to implicitly integrate the index matrices of them. We formulate the index embedding as an optimization problem from the perspective of neighborhood sharing and solve it with an alternating index update scheme. After the iterative embedding, only the embedded CNN index is kept for on-line query, which demonstrates significant gain in retrieval accuracy, with very economical memory cost. Extensive experiments have been conducted on the public datasets with million-scale distractor images. The experimental results reveal that, compared with the recent state-of-the-art retrieval algorithms, our approach achieves competitive accuracy performance with less memory overhead and efficient query computation.