Data-stream processing has continuously risen in importance as the amount of available data has been steadily increas- ing over the last decade. Besides traditional domains such as data-center monitoring and click analytics, there is an increasing number of network-enabled production machines that generate continuous streams of data. Due to their continuous nature, queries on data-streams can be more complex, and distinctly harder to understand then database queries. As users have to consider operational details, maintenance and debugging become challenging. Current approaches model data-streams as sequences, be- cause this is the way they are physically received. These models result in an implementation-focused perspective. We explore an alternate way of modeling data-streams by focusing on time-slicing semantics. This focus results in a model based on functions, which is better suited for reasoning about query semantics. By adapting the definitions of relevant concepts in stream processing to our model, we illustrate the practical useful- ness of our approach. Thereby, we link data-streams and query primitives to concepts in functional programming and mathematics. Most noteworthy, we prove that data-streams are monads, and show how to derive monad definitions for current data-stream models. We provide an abstract, yet practical perspective on data- stream related subjects based on a sound, consistent query model. Our work can serve as solid foundation for future data-stream query-languages.
We are interested in detecting episodes in a data stream that are characterized by a period of time over which a condition holds, usually with a minimum duration. For example, we might want to know whenever any router has a packet-drop rate above 0.3 continuously for more than two minutes. Such episodes can be interesting in their own right for monitoring purposes, but they can also specify target regions for examination over the original or other stream. For instance, for each router-drop episode we detect, we might want to count the number of control messages the router received. We assert the key requirements are to detect the episodes, detect them accurately, and detect them promptly. Current capabilities for data-stream management systems (DSMSs) include functionality, such as pattern-matching and windowed aggregates, that can help with detecting some kinds of episodes. We offer a third alternative, frames, which generalizes the other two. Frames are intervals that segment a data stream into regions of interest. In contrast to windows, frame boundaries can be data dependent, such as when a predicate holds for a given duration, or the maximum and minimum values of an attribute diverge more than a certain amount. We introduce frames and their theory, plus their implementation in the NiagaraST DSMS. We then demonstrate some advantages of frames versus windows, such as better characterization of episodes, on real data sets and explore an extension, fragments, to deal with long episodes.