Much of the worlds data is streaming, time-series data, where anomalies give significant information in critical situations. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, and learn while simultaneously making predictions. We present a novel anomaly detection technique based on an on-line sequence memory algorithm called Hierarchical Temporal Memory (HTM). We show results from a live application that detects anomalies in financial metrics in real-time. We also test the algorithm on NAB, a published benchmark for real-time anomaly detection, where our algorithm achieves best-in-class results.
There have been a number of model-based approaches applied to specific domains. These tend to be extremely specific to the domain they are modeling. Examples include anomaly detection in aircraft engine measurements @cite_1 , cloud datacenter temperatures @cite_4 , and ATM fraud detection @cite_2 . While these approaches may have success in a specific domain, they are not suitable for general purpose applications.
@cite_1 @cite_4 @cite_2
2101623967 2012054765 2040328877
This paper presents a model-based anomaly detection architecture designed for analyzing streaming transient aircraft engine measurement data. The technique calculates and monitors residuals between sensed engine outputs and model predicted outputs for anomaly detection purposes. Pivotal to the performance of this technique is the ability to construct a model that accurately reflects the nominal operating performance of the engine. The dynamic model applied in the architecture is a piecewise linear design comprising steady-state trim points and dynamic state space matrices. A simple curve-fitting technique for updating the model trim point information based on steady-state information extracted from available nominal engine measurement data is presented. Results from the application of the model-based approach for processing actual engine test data are shown. These include both nominal fault-free test case data and seeded fault test case data. The results indicate that the updates applied to improve the model trim point information also improve anomaly detection performance. Recommendations for follow-on enhancements to the technique are also presented and discussed.© 2014 ASME The growing importance, large scale, and high server density of high-performance computing datacenters make them prone to strategic attacks, misconfigurations, and failures (cooling as well as computing infrastructure). Such unexpected events lead to thermal anomalies - hotspots, fugues, and coldspots - which significantly impact the total cost of operation of datacenters. A model-based thermal anomaly detection mechanism, which compares expected (obtained using heat generation and extraction models) and observed thermal maps (obtained using thermal cameras) of datacenters is proposed. In addition, a Thermal Anomaly-aware Resource Allocation (TARA) scheme is designed to create time-varying thermal fingerprints of the datacenter so to maximize the accuracy and minimize the latency of the aforementioned model-based detection. TARA significantly improves the performance of model-based anomaly detection compared to state-of-the-art resource allocation schemes. Model-based anomaly detection in technical systems is an important application field of artificial intelligence. We consider discrete event systems, which is a system class to which a wide range of relevant technical systems belong and for which no comprehensive model-based anomaly detection approach exists so far. The original contributions of this paper are threefold: First, we identify the types of anomalies that occur in discrete event systems and we propose a tailored behavior model that captures all anomaly types, called probabilistic deterministic timed-transition automata (PDTTA). Second, we present a new algorithm to learn a PDTTA from sample observations of a system. Third, we describe an approach to detect anomalies based on a learned PDTTA. An empirical evaluation in a practical application, namely ATM fraud detection, shows promising results.