In addition to the frequency of terms in a document collection, the distribution of terms plays an important role in determining the relevance of documents. In this paper, a new approach for representing term positions in documents is presented. The approach allows an efficient evaluation of term-positional information at query evaluation time. Three applications are investigated: a function-based ranking optimization representing a user-defined document region, a query expansion technique based on overlapping the term distributions in the top-ranked documents, and cluster analysis of terms in documents. Experimental results demonstrate the effectiveness of the proposed approach.
One of the first approaches using abstract representations of term distributions in documents is Fourier Domain Scoring (FDS), proposed by @cite_8 . FDS performs a separate magnitude and phase analysis of term position signals to produce an optimized ranking. It creates an index based on page segmentation, storing term frequency and approximated positions in the document. FDS processes the indexed data using the to perform the corresponding spectral analysis.
@cite_8
2105660794
Current document retrieval methods use a vector space similarity measure to give scores of relevance to documents when related to a specific query. The central problem with these methods is that they neglect any spatial information within the documents in question. We present a new method, called Fourier Domain Scoring (FDS), which takes advantage of this spatial information, via the Fourier transform, to give a more accurate ordering of relevance to a document set. We show that FDS gives an improvement in precision over the vector space similarity measures for the common case of Web like queries, and it gives similar results to the vector space measures for longer queries.