Complex event processing (CEP) is widely employed to detect occurrences of predefined combinations (patterns) of events in massive data streams. As new events are accepted, they are matched using some type of evaluation structure, commonly optimized according to the statistical properties of the data items in the input stream. However, in many real-life scenarios the data characteristics are never known in advance or are subject to frequent on-the-fly changes. To modify the evaluation structure as a reaction to such changes, adaptation mechanisms are employed. These mechanisms typically function by monitoring a set of properties and applying a new evaluation plan when significant deviation from the initial values is observed. This strategy often leads to missing important input changes or it may incur substantial computational overhead by over-adapting. In this paper, we present an efficient and precise method for dynamically deciding whether and how the evaluation structure should be reoptimized. This method is based on a small set of constraints to be satisfied by the monitored values, defined such that a better evaluation plan is guaranteed if any of the constraints is violated. To the best of our knowledge, our proposed mechanism is the first to provably avoid false positives on reoptimization decisions. We formally prove this claim and demonstrate how our method can be applied on known algorithms for evaluation plan generation. Our extensive experimental evaluation on real-world datasets confirms the superiority of our strategy over existing methods in terms of performance and accuracy.
Complex event processing is an increasingly active research field @cite_44 . The origins of CEP systems can be traced to older data stream managements systems (DSMSs), including Aurora Borealis @cite_4 , Stream @cite_9 , TelegraphCQ @cite_13 , and NiagaraCQ @cite_42 . This was followed by the emergence of a broad variety of solutions for detecting occurrences of situations of interest, as opposed to generic data, including frameworks such as SASE SASE+ @cite_11 @cite_15 @cite_47 , CEDR @cite_12 , Cayuga @cite_27 , T-Rex @cite_41 and Amit @cite_1 . Esper @cite_14 and IBM System S @cite_2 are examples of widely used commercial CEP providers.
@cite_47 @cite_14 @cite_4 @cite_41 @cite_9 @cite_42 @cite_1 @cite_44 @cite_27 @cite_2 @cite_15 @cite_13 @cite_12 @cite_11
2094206427 1996418875 2043390121 2230878985
Complex Event Processing (CEP) has emerged as a technology for monitoring event streams in search of user specified event patterns. When a CEP system is deployed in sensitive environments the user may wish to mitigate leaks of private information while ensuring that useful nonsensitive patterns are still reported. In this paper we consider how to suppress events in a stream to reduce the disclosure of sensitive patterns while maximizing the detection of nonsensitive patterns. We first formally define the problem of utility-maximizing event suppression with privacy preferences, and analyze its computational hardness. We then design a suite of real-time solutions to solve this problem. Our first solution optimally solves the problem at the event-type level. The second solution, at the event-instance level, further optimizes the event-type level solution by exploiting runtime event distributions using advanced pattern match cardinality estimation techniques. Our user study and experimental evaluation over both real-world and synthetic event streams show that our algorithms are effective in maximizing utility yet still efficient enough to offer near real-time system responsiveness. Several application domains involve detecting complex situations and reacting to them. This asks for a Complex Event Processing (CEP) middleware specifically designed to timely process large amounts of event notifications as they flow from the peripheral to the center of the system, to identify the composite events relevant for the application. To answer this need we designed T-Rex, a new CEP middleware that combines expressiveness and efficiency. On the one hand, it adopts a language (TESLA) explicitly conceived to easily and naturally describe composite events. On the other hand, it provides an efficient event detection algorithm based on automata to interpret TESLA rules. Our evaluation shows that the T-Rex engine can process a large number of complex rules with a reduced overhead, even in the presence of challenging workloads. Detecting complex patterns in event streams, i.e., complex event processing (CEP), has become increasingly important for modern enterprises to react quickly to critical situations. In many practical cases business events are generated based on pre-defined business logics. Hence constraints, such as occurrence and order constraints, often hold among events. Reasoning using these known constraints enables us to predict the non-occurrences of certain future events, thereby helping us to identify and then terminate the long running query processes that are guaranteed to not lead to successful matches. In this work, we focus on exploiting event constraints to optimize CEP over large volumes of business transaction streams. Since the optimization opportunities arise at runtime, we develop a runtime query unsatisfiability (RunSAT) checking technique that detects optimal points for terminating query evaluation. To assure efficiency of RunSAT checking, we propose mechanisms to precompute the query failure conditions to be checked at runtime. This guarantees a constant-time RunSAT reasoning cost, making our technique highly scalable. We realize our optimal query termination strategies by augmenting the query with Event-Condition-Action rules encoding the pre-computed failure conditions. This results in an event processing solution compatible with state-of-the-art CEP architectures. Extensive experimental results demonstrate that significant performance gains are achieved, while the optimization overhead is small. Complex Event Processing (CEP) detects complex events or patterns of event sequences based on a set of rules defined by a domain expert. However, it lowers the reliability of a system as the set of rules defined by an expert changes along with dynamic changes in the domain environment. A human error made by an expert is another factor that may undermine the reliability of the system. In an effort to address such problems, this study introduces Collaborative Rule Mining Engine (CRME) designed to automatically mine rules based on the history of decisions made by a domain expert by adopting a collaborative filtering approach, which is effective in mimicking and predicting human decision-making in an environment where there are sufficient data or information to do so. Furthermore, this study suggests an adaptive CEP technique, which does not hamper the reliability since it prevents potential errors caused by mistakes of domain experts and adapts to changes in the domain environment on its own as it is linked to the system proposed by Bharagavi [10]. In a bid to verify this technique, an automated stocks trading system will be established and its performance will be measured using the rate of return.