Outdoor visual localization is a crucial component to many computer vision systems. We propose an approach to localization from images that is designed to explicitly handle the strong variations in appearance happening between daytime and nighttime. As revealed by recent long-term localization benchmarks, both traditional feature-based and retrieval-based approaches still struggle to handle such changes. Our novel localization method combines a state-of-the-art image retrieval architecture with condition-specific sub-networks allowing the computation of global image descriptors that are explicitly dependent of the capturing conditions. We show that our approach improves localization by a factor of almost 300 compared to the popular VLAD-based methods on nighttime localization.
Compared to 3D feature-based methods, retrieval-based methods offer two key advantages. First, the extension of such methods to city-scale scenarios is trivial @cite_70 . Besides, in a very large database, unsupervised descriptor compression like PCA @cite_16 or Product Quantization (PQ) @cite_51 enables efficient approximate nearest-neighbour search with little loss in performance @cite_74 . Secondly, evaluations in day-to-night @cite_62 @cite_9 or long-term @cite_10 @cite_0 visual localization reveal that retrieval-based methods outperform other feature-based methods like Active Search @cite_5 at a coarse precision level. Image pre-processing @cite_14 or translation @cite_24 can be employed to bring all images to a visually similar and condition-invariant representation. However, image translation methods are complex to train, require a full retraining for every condition and add to the overall computational cost of retrieval methods. Our method outperforms both retrieval and feature-based techniques without the need for such pre-processing.
@cite_62 @cite_14 @cite_70 @cite_10 @cite_9 @cite_0 @cite_24 @cite_74 @cite_5 @cite_16 @cite_51
2103924867 2135364649 2153335008 2342423902
The objective of this paper is large scale object instance retrieval, given a query image. A starting point of such systems is feature detection and description, for example using SIFT. The focus of this paper, however, is towards very large scale retrieval where, due to storage requirements, very compact image descriptors are required and no information about the original SIFT descriptors can be accessed directly at run time. We start from VLAD, the state-of-the art compact descriptor introduced by for this purpose, and make three novel contributions: first, we show that a simple change to the normalization method significantly improves retrieval performance, second, we show that vocabulary adaptation can substantially alleviate problems caused when images are added to the dataset after initial vocabulary learning. These two methods set a new state-of-the-art over all benchmarks investigated here for both mid-dimensional (20k-D to 30k-D) and small (128-D) descriptors. Our third contribution is a multiple spatial VLAD representation, MultiVLAD, that allows the retrieval and localization of objects that only extend over a small part of an image (again without requiring use of the original image SIFT descriptors). Recently two lines of image retrieval algorithms demonstrate excellent scalability: 1) local features indexed by a vocabulary tree, and 2) holistic features indexed by compact hashing codes. Although both of them are able to search visually similar imageseffectively, their retrieval precision may vary dramatically among queries. Therefore, combining these two types of methods is expected to further enhance the retrieval precision. However, the feature characteristics and the algorithmic procedures of these methods are dramatically different, which is very challenging for the feature-level fusion. This motivates us to investigate how to fuse the ordered retrieval sets, i.e., the ranks of images, given by multiple retrieval methods, to boost the retrieval precision without sacrificing theirscalability. In this paper, we model retrieval ranks as graphs of candidate images and propose a graph-based query specific fusion approach, where multiple graphs are merged and reranked by conducting a link analysis on a fused graph. The retrieval quality of an individual method is measured on-the-fly by assessing the consistency of the top candidates’ nearest neighborhoods. Hence, it iscapable of adaptively integrating the strengths of the retrieval methods using local or holistic features for different query images. This proposed method does not need any supervision, has few parameters, and is easy to implement. Extensive and thorough experiments have been conducted on four public datasets, i.e., the UKbench , Corel-5K , Holidays and the large-scale San Francisco Landmarks datasets. Our proposed method has achieved very competitive performance, including state-of-the-art results on several data sets, e.g., the N-S score 3.83 for UKbench . With broader availability of large-scale 3D model repositories, the need for efficient and effective exploration becomes more and more urgent. Existing model retrieval techniques do not scale well with the size of the database since often a large number of very similar objects are returned for a query, and the possibilities to refine the search are quite limited. We propose an interactive approach where the user feeds an active learning procedure by labeling either entire models or parts of them as “like” or “dislike” such that the system can automatically update an active set of recommended models. To provide an intuitive user interface, candidate models are presented based on their estimated relevance for the current query. From the methodological point of view, our main contribution is to exploit not only the similarity between a query and the database models but also the similarities among the database models themselves. We achieve this by an offline pre-processing stage, where global and local shape descriptors are computed for each model and a sparse distance metric is derived that can be evaluated efficiently even for very large databases. We demonstrate the effectiveness of our method by interactively exploring a repository containing over 100 K models. Local descriptors are mostly extracted over the raw intensity values of the image and requires the analysis of patterns in local pixel neighborhoods. The raw images don't provide much local relationship information required for the local descriptors. To resolve this problem, a new method of local image feature description is proposed in this paper based on the patterns extracted from the multiple filtered images. The whole process is structured as follows: First, the images are filtered with the bag of filters (BoF) and then local binary pattern (LBP) is computed over each filtered image and finally concatenated to find out the single BoF-LBP descriptor. The local information enriches in the filtered images improves the discriminative ability of the final descriptor. Content based image retrieval (CBIR) experiments are performed to observe the effectiveness of the proposed approach and compared with the recent state-of-the-art methods. The experiments are conducted over four benchmark databases having natural and texture images such as Corel-1k, Corel-10k, MIT-VisTex and STex-512S. The experimental results confirm that the introduced approach is able to improve the retrieval performance of the CBIR system.