Given a large dataset of binary codes and a binary query point, we address how to efficiently find @math codes in the dataset that yield the largest cosine similarities to the query. The straightforward answer to this problem is to compare the query with all items in the dataset, but this is practical only for small datasets. One potential solution to enhance the search time and achieve sublinear cost is to use a hash table populated with binary codes of the dataset and then look up the nearby buckets to the query to retrieve the nearest neighbors. However, if codes are compared in terms of cosine similarity rather than the Hamming distance, then the main issue is that the order of buckets to probe is not evident. To examine this issue, we first elaborate on the connection between the Hamming distance and the cosine similarity. Doing this allows us to systematically find the probing sequence in the hash table. However, solving the nearest neighbor search with a single table is only practical for short binary codes. To address this issue, we propose the angular multi-index hashing search algorithm which relies on building multiple hash tables on binary code substrings. The proposed search algorithm solves the exact angular @math nearest neighbor problem in a time that is often orders of magnitude faster than the linear scan baseline and even approximation methods.
A classical paradigm to reduce the computational cost of the nearest neighbor search relies on tree-based indexing structures which offer logarithmic query time, @math , on the average. Perhaps the best known example of such techniques is the kd-tree @cite_4 with a worst-case search time of @math . Following @cite_4 , many other tree-based indexing algorithms have been proposed (see @cite_22 for an overview). However, these methods suffer from a phenomenon commonly known as the curse of dimensionality . This means, as the number of dimensions increases, these methods quickly reduce to the exhaustive search. Typically, the kd-tree and its variants are efficient for dimensions less than 20 @cite_44 @cite_14 .
@cite_44 @cite_14 @cite_4 @cite_22
1502916507 2165558283 2109064983 2045622468
The nearestor near-neighbor query problems arise in a large variety of database applications, usually in the context of similarity searching. Of late, there has been increasing interest in building search index structures for performing similarity search over high-dimensional data, e.g., image databases, document collections, time-series databases, and genome databases. Unfortunately, all known techniques for solving this problem fall prey to the of dimensionality." That is, the data structures scale poorly with data dimensionality; in fact, if the number of dimensions exceeds 10 to 20, searching in k-d trees and related structures involves the inspection of a large fraction of the database, thereby doing no better than brute-force linear search. It has been suggested that since the selection of features and the choice of a distance metric in typical applications is rather heuristic, determining an approximate nearest neighbor should su ce for most practical purposes. In this paper, we examine a novel scheme for approximate similarity search based on hashing. The basic idea is to hash the points Supported by NAVY N00014-96-1-1221 grant and NSF Grant IIS-9811904. Supported by Stanford Graduate Fellowship and NSF NYI Award CCR-9357849. Supported by ARO MURI Grant DAAH04-96-1-0007, NSF Grant IIS-9811904, and NSF Young Investigator Award CCR9357849, with matching funds from IBM, Mitsubishi, Schlumberger Foundation, Shell Foundation, and Xerox Corporation. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and or special permission from the Endowment. Proceedings of the 25th VLDB Conference, Edinburgh, Scotland, 1999. from the database so as to ensure that the probability of collision is much higher for objects that are close to each other than for those that are far apart. We provide experimental evidence that our method gives signi cant improvement in running time over other methods for searching in highdimensional spaces based on hierarchical tree decomposition. Experimental results also indicate that our scheme scales well even for a relatively large number of dimensions (more than 50). This paper develops the multidimensional binary search tree (or k -d tree, where k is the dimensionality of the search space) as a data structure for storage of information to be retrieved by associative searches. The k -d tree is defined and examples are given. It is shown to be quite efficient in its storage requirements. A significant advantage of this structure is that a single data structure can handle many types of queries very efficiently. Various utility algorithms are developed; their proven average running times in an n record file are: insertion, O (log n ); deletion of the root, O ( n ( k -1) k ); deletion of a random node, O (log n ); and optimization (guarantees logarithmic performance of searches), O ( n log n ). Search algorithms are given for partial match queries with t keys specified [proven maximum running time of O ( n ( k - t ) k )] and for nearest neighbor queries [empirically observed average running time of O (log n ).] These performances far surpass the best currently known algorithms for these tasks. An algorithm is presented to handle any general intersection query. The main focus of this paper is theoretical. It is felt, however, that k -d trees could be quite useful in many applications, and examples of potential uses are given. A method is presented that uses an approximate nearest neighbor method for determining correspondences within the iterative closest point algorithm. The method is based upon the k-d tree. The standard k-d tree uses a tentative backtracking search to identify nearest neighbors. In contrast, the approximate k-d tree (Ak-d tree) applies a depth-first nontentative search to the k-d tree structure. This search improves runtime efficiency, with the tradeoff of reducing the accuracy of the determined correspondences. This approximate search is applied to early iterations of the iterative closest point algorithm, transitioning to the standard k-d tree for the final iterations after the change in the mean square error of the correspondences becomes sufficiently small. The method benefits both from the improved time performance of the approximate search in early iterations as well as the full accuracy of the complete search in later iterations. Experimental results indicate that the time efficiency of Ak-d tree is superior to the k-d tree and Elias for moderately large point sets. The change in the shape of the minimum potential well space is subtle, and the convergence properties are often identical. In those cases where the global minimum was not achieved, the difference in final mse was very small. In one trial, Ak-d tree converged faster to a better minimum with a smaller mse, which indicates that the use of approximate methods may be beneficial in the presence of outliers. Conventional spatial queries, such as range search and nearest neighbor retrieval, involve only conditions on objects' geometric properties. Today, many modern applications call for novel forms of queries that aim to find objects satisfying both a spatial predicate, and a predicate on their associated texts. For example, instead of considering all the restaurants, a nearest neighbor query would instead ask for the restaurant that is the closest among those whose menus contain “steak, spaghetti, brandy” all at the same time. Currently, the best solution to such queries is based on the IR 2-tree, which, as shown in this paper, has a few deficiencies that seriously impact its efficiency. Motivated by this, we develop a new access method called the spatial inverted index that extends the conventional inverted index to cope with multidimensional data, and comes with algorithms that can answer nearest neighbor queries with keywords in real time. As verified by experiments, the proposed techniques outperform the IR 2-tree in query response time significantly, often by a factor of orders of magnitude.