Data-stream processing has continuously risen in importance as the amount of available data has been steadily increas- ing over the last decade. Besides traditional domains such as data-center monitoring and click analytics, there is an increasing number of network-enabled production machines that generate continuous streams of data. Due to their continuous nature, queries on data-streams can be more complex, and distinctly harder to understand then database queries. As users have to consider operational details, maintenance and debugging become challenging. Current approaches model data-streams as sequences, be- cause this is the way they are physically received. These models result in an implementation-focused perspective. We explore an alternate way of modeling data-streams by focusing on time-slicing semantics. This focus results in a model based on functions, which is better suited for reasoning about query semantics. By adapting the definitions of relevant concepts in stream processing to our model, we illustrate the practical useful- ness of our approach. Thereby, we link data-streams and query primitives to concepts in functional programming and mathematics. Most noteworthy, we prove that data-streams are monads, and show how to derive monad definitions for current data-stream models. We provide an abstract, yet practical perspective on data- stream related subjects based on a sound, consistent query model. Our work can serve as solid foundation for future data-stream query-languages.
In our earlier work @cite_8 , we focused on a generic set of operations for continuous queries. We motivated the necessity for pattern matching and more powerful window constructs, and showed that equality may be proven. However, we could not provide a formal proof, as our model relied on conventions, and a sequential perspective is less convenient for proofs than a functional one. This is one of the main motivations for this paper.
@cite_8
2028056304 2141957180 1626378195 113905406
Many interesting applications of continuous-query processing are concerned with pattern matching or complex temporal aggregation of events. Real-world queries that rely on these operations are difficult to implement in current stream-processing systems. The reason seems to be a gap between two types of existing query languages: Some languages (e. g. CQL) offer a small set of simple operators that can be combined in order to create complex queries. While these languages provide sound and comprehensible semantics, they lack the expressiveness required for many real-world applications. Other approaches (e. g. Aurora) provide powerful operators but lack semantic strictness, which is required for reasoning about query results. Such reasoning is a prerequisite for safe query optimization. We try to bridge this gap by integrating operators for pattern matching and time-aware aggregates into a general-purpose stream model featuring stream partitioning. These operators can answer several questions that we have found to be relevant in a real-world object-tracking scenario. Moreover, they are formally defined, allowing expressive and efficient queries to be written in CQL-like languages, while remaining understandable and easy to use. Given a set @math of @math strings of total length @math , our task is to report the "most relevant"strings for a given query pattern @math . This involves somewhat more advanced query functionality than the usual pattern matching, as some notion of "most relevant" is involved. In information retrieval literature, this task is best achieved by using inverted indexes. However, inverted indexes work only for some predefined set of patterns. In the pattern matching community, the most popular pattern-matching data structures are suffix trees and suffix arrays. However, a typical suffix tree search involves going through all the occurrences of the pattern over the entire string collection, which might be a lot more than the required relevant documents. The first formal framework to study such kind of retrieval problems was given by [Muthukrishnan, 2002]. He considered two metrics for relevance: frequency and proximity. He took a threshold-based approach on these metrics and gave data structures taking @math words of space. We study this problem in a slightly different framework of reporting the top @math most relevant documents (in sorted order) under similar and more general relevance metrics. Our framework gives linear space data structure with optimal query times for arbitrary score functions. As a corollary, it improves the space utilization for the problems in [Muthukrishnan, 2002] while maintaining optimal query performance. We also develop compressed variants of these data structures for several specific relevance metrics. Matching dependencies were recently introduced as declarative rules for data cleaning and entity resolution. Enforcing a matching dependency on a database instance identifies the values of some attributes for two tuples, provided that the values of some other attributes are sufficiently similar. Assuming the existence of matching functions for making two attributes values equal, we formally introduce the process of cleaning an instance using matching dependencies, as a chase-like procedure. We show that matching functions naturally introduce a lattice structure on attribute domains, and a partial order of semantic domination between instances. Using the latter, we define the semantics of clean query answering in terms of certain possible answers as the greatest lower bound least upper bound of all possible answers obtained from the clean instances. We show that clean query answering is intractable in some cases. Then we study queries that behave monotonically w.r.t. semantic domination order, and show that we can provide an under over approximation for clean answers to monotone queries. Moreover, non-monotone positive queries can be relaxed into monotone queries. We recently proposed a data structure, called associative-commutative discrimination nets, that supports efficient algorithms for (many-to-one) term matching in the presence of associative-commutative functions. In this paper we discuss the integration of such discrimination nets into an actual equational theorem prover and report on corresponding experiments. The general associative-commutative matching problem is known to be NP-complete, but can be solved in polynomial time if the given terms are linear, i.e., do not contain multiple occurrences of the same variable. We therefore have implemented a two-stage matching procedure. First we check whether a match exists for the linearized versions of the given terms (where different occurrences of the same variable are replaced by different new variables). If a match for the linearized terms does exist, we then determine whether there is also a match for the original, non-linear terms (i.e., whether the proposed substitutions for different occurrences of the same variable are consistent). Our experimental results indicate that this approach works very well in theorem proving, where most matching attempts actually fail and are filtered out during the first stage, so that the second, more expensive stage of the algorithm is only needed in comparatively few cases.