This paper addresses the problem of determining the distance between two regular languages. It will show how to expand Jaccard distance, which works on finite sets, to potentially-infinite regular languages. The entropy of a regular language plays a large role in the extension. Much of the paper is spent investigating the entropy of a regular language. This includes addressing issues that have required previous authors to rely on the upper limit of Shannon's traditional formulation of entropy, because its limit does not always exist. The paper also includes proposing a new limit based formulation for the entropy of a regular language and proves that formulation to both exist and be equivalent to Shannon's original formulation (when it exists). Additionally, the proposed formulation is shown to equal an analogous but formally quite different notion of topological entropy from Symbolic Dynamics -- consequently also showing Shannon's original formulation to be equivalent to topological entropy. Surprisingly, the natural Jaccard-like entropy distance is trivial in most cases. Instead, the entropy sum distance metric is suggested, and shown to be granular in certain situations.
More recently, Cui directly address distances between regular languages using a generalization of Jaccard distance @cite_14 . That paper usefully expands the concept of Jaccard distance to regular languages by (1) using entropy to handle infinite sized regular languages (they use the upper limit notion of entropy described above), and (2) allowing operations other than intersection to be used in the numerator. Further, Cui suggest and prove properties of several specific distance functions between regular languages. The distance functions in this paper do not generalize the Jaccard distance in the same way, but are proven to be metrics or pseudo-metrics.
@cite_14
2144638632 1998314653 2144221002 2099812803
We show how appropriately chosen functions f which we call distinguishing can be used to make deterministic finite automata backward deterministic. This idea can be exploited to design regular language classes called f-distinguishable which are identifiable in the limit from positive samples. Special cases of this approach are the k-reversible and terminal distinguishable languages, as discussed in Angluin (J. Assoc. Comput. Mach. 29 (3) (1982) 741), Fernau (Technical Report WSI-99-23, Universitat Tubingen (Germany), Wilhelm-Schickard-Institut fur Informatik, 1999, Short version published in the proceedings of AMAI 2000, see http: rutcor.rutgers.edu amai aimath00 AcceptedCont.htm, Proc. 15th Internat. Conf. on Pattern Recognition (ICPR 2000), Vol. 2, IEEE Press, New York, 2000, pp. 125?128), Radhakrishnan (Ph.D. Thesis, Department of Computer Science and Engineering, Indian Institute of Technology, Bombay, India, 1987), Radhakrishnan and Nagaraja (IEEE Trans. Systems, Man Cybernet. 17 (6) (1987) 982). Moreover, we show that all regular languages may be approximated in the setting introduced by Kobayashi and Yokomori (in: K. P. Jantke, T. Shinohara, Th. Zeugmann (Eds.), Proc. Sixth Internat. Conf. Algorithmic Learning Theory (ALT?95), Lecture Notes in Computer Science Lecture Notes in Artificial Intelligence, Vol. 997, Springer, Berlin, 1995, pp. 298?312), (Theoret. Comput. Sci. 174 (1997) 251?257) by any class of f-distinguishable languages. Observe that the class of all function-distinguishable languages is equal to the class of regular languages. We use an information-theoretic notion, namely, (Shannon) information rate, to generalize common syntactic similarity metrics (like Hamming distance and longest common subsequences) between strings to ones between languages. We show that the similarity metrics between two regular languages are computable. We further study self-similarity of a regular language under various similarity metrics. As far as semantic similarity is concerned, we study the amplitude of an automaton, which intuitively characterizes how much a typical execution of the automaton fluctuates. Finally, we investigate, through experiments, how to measure similarity between two real-world programs using Lempel-Ziv compression on the runs at the assembly level. A new class of distances appropriate for measuring similarity relations between sequences, say one type of similarity per distance, is studied. We propose a new "normalized information distance," based on the noncomputable notion of Kolmogorov complexity, and show that it is in this class and it minorizes every computable distance in the class (that is, it is universal in that it discovers all computable similarities). We demonstrate that it is a metric and call it the similarity metric . This theory forms the foundation for a new practical tool. To evidence generality and robustness, we give two distinctive applications in widely divergent areas using standard compression programs like gzip and GenCompress. First, we compare whole mitochondrial genomes and infer their evolutionary history. This results in a first completely automatic computed whole mitochondrial phylogeny tree. Secondly, we fully automatically compute the language tree of 52 different languages. We introduce a new probabilistic proximity search algorithm for range and A"-nearest neighbor (A"-NN) searching in both coordinate and metric spaces. Although there exist solutions for these problems, they boil down to a linear scan when the space is intrinsically high dimensional, as is the case in many pattern recognition tasks. This, for example, renders the A"-NN approach to classification rather slow in large databases. Our novel idea is to predict closeness between elements according to how they order their distances toward a distinguished set of anchor objects. Each element in the space sorts the anchor objects from closest to farthest to it and the similarity between orders turns out to be an excellent predictor of the closeness between the corresponding elements. We present extensive experiments comparing our method against state-of-the-art exact and approximate techniques, both in synthetic and real, metric and nonmetric databases, measuring both CPU time and distance computations. The experiments demonstrate that our technique almost always improves upon the performance of alternative techniques, in some cases by a wide margin.