Hoeffding Tree Algorithms for Anomaly Detection in Streaming Datasets: A Survey <s> Introduction <s> Data streams, which can be considered as one of the primary sources of what is called big data, arrive continuously with high speed. The biggest challenge in data streams mining is to deal with concept drifts, during which ensemble methods are widely employed. The ensembles for handling concept drift can be categorized into two different approaches: online and block-based approaches. The primary disadvantage of the block-based ensembles lies in the difficulty of tuning the block size to provide a tradeoff between fast reactions to drifts. Motivated by this challenge, we put forward an online ensemble paradigm, which aims to combine the best elements of block-based weighting and online processing. The algorithm uses the adaptive windowing as a change detector. Once a change is detected, a new classifier is built replacing the worst one in the ensemble. By experimental evaluations on both synthetic and real-world datasets, our method performs significantly better than other ensemble approaches. <s> BIB001 </s> Hoeffding Tree Algorithms for Anomaly Detection in Streaming Datasets: A Survey <s> Introduction <s> High-dimensional data streams are becoming increasingly ubiquitous in industrial systems. Efficient detection of system faults from these data can ensure the reliability and safety of the system. The difficulties brought about by high dimensionality and data streams are mainly the “curse of dimensionality” and concept drifting, and one current challenge is to simultaneously address them. To this purpose, this paper presents an approach to fault detection from nonstationary high-dimensional data streams. An angle-based subspace anomaly detection approach is proposed to detect low-dimensional subspace faults from high-dimensional datasets. Specifically, it selects fault-relevant subspaces by evaluating vectorial angles and computes the local outlier-ness of an object in its subspace projection. Based on the sliding window strategy, the approach is further extended to an online mode that can continuously monitor system states. To validate the proposed algorithm, we compared it with the local outlier factor-based approaches on artificial datasets and found the algorithm displayed superior accuracy. The results of the experiment demonstrated the efficacy of the proposed algorithm. They also indicated that the algorithm has the ability to discriminate low-dimensional subspace faults from normal samples in high-dimensional spaces and can be adaptive to the time-varying behavior of the monitored system. The online subspace learning algorithm for fault detection would be the main contribution of this paper. <s> BIB002 </s> Hoeffding Tree Algorithms for Anomaly Detection in Streaming Datasets: A Survey <s> Introduction <s> Electricity transmission systems are networked cyber physical systems that are subject to many well-known control, weather, and equipment failure related contingencies which can disrupt power delivery. Cyber-attacks against electric transmission systems are another class of contingency which can disrupt power delivery. Wide area monitoring systems (WAMSs) enhanced with phasor measurement units provide high volume and high velocity power system sensor data which can be combined with traditional power system data sources and cyber data sources to enable real time detection of both types of contingencies. This paper describes research toward a cyber-power event and intrusion detection system (EIDS) which can be used for multiclass or binary-class classification of traditional power system contingencies and cyber-attacks. The continuous streams of high speed data from WAMS pose significant challenges in data storage, management, and handling. Data stream mining addresses the continuous data problem and can deal with very large data sizes. Hoeffding adaptive trees (HAT) augmented with the drift detection method (DDM) and adaptive windowing (ADWIN) can effectively be used to classify traditional and cyber contingencies in real time. Experiments performed for this paper demonstrate HAT + DDM + ADWIN provides classification accuracy of greater than 94% for multiclass and greater than 98% for binary class classification for a dataset with artifacts from 45 classes of cyber-power contingencies. Results also show that HAT + DDM + ADWIN has a small memory foot print and a fast evaluation time which enables real time EIDS. <s> BIB003
A wide variety of application domains use streaming data. These domains in-How to cite this paper: Muallem, A., Shetty, S., Pan, J.W., Zhao, J. and Biswal, B. In most scenarios where new, dynamic data is continuously generated, streaming data is beneficial. Data collection of information is the first step in streaming data and it can evolve into a more sophisticated real-time processing. Initially these applications may perform simple data analysis with simple actions as a response in which may ultimately evolve into more sophisticated types of data analysis, such as applying machine learning algorithms to extract precise information from the data. There are several challenges on data mining algorithm design posed by data streams  . One is the limited use of resources  . Most big data, in which data are produced continuously, can be recorded as data streams BIB001 . Many machine learning algorithms have been developed for sophisticated data analysis on streaming data. These include classification, regression, and clustering algorithms. The model in the data streams is that it must address the problem of three features of big data: large volume, large velocity, and large variety. A fundamental model in data streams is the necessary of dealing with data whose nature or distribution changes over time  . Strategies for detecting and quantifying change, forgetting stale experiments, and model revision is required in dealing with time-changing data  . Decision trees are a type of classifier algorithms   . A decision tree is learned top-down by recursively replacing leaves by test nodes, starting at the root  . All available attributes are compared and choosing the best one according to some heuristic measure is how the attribute at a node is tested  . Classical decision tree learners are severely limited in the number of examples they can learn from, since they assume that all training examples can be stored simultaneously in memory  . Hoeffding Trees, an incremental, anytime decision tree induction algorithm capable of learning from massive data streams, was developed by Domingos and Hulten   . The fact that a small sample can often be enough to choose an optimal splitting attribute is the theory of Hoeffding Trees  . The Hoeffding bound mathematically supports this idea by quantifying the number of observations or examples needed to estimate some statistics within a prescribed decision or goodness of an attribute  . The Hoeffding Trees have sound guarantees of performance, a theoretically interesting feature not shared by other incremental decision tree learners. Figure 1 provides the Hoeffding Tree Induction Algorithm referenced from  . Anomaly detection for streaming data is an important topic of research due to its nature of detecting vital information. This vital information can include intrusion and other failure information. There is extensive work on anomaly detection techniques BIB002 [8]  for streaming data. These techniques look into fault detection and intrusion detection by exploring methods for identifying anomalies based on the scalability and generality of the data streams, they do not can be created by cyber-attacks BIB003 . According to data mining, the target information that a model is trying to predict are the concepts BIB003 . The change of the underlying concept over time is known as concept change. BIB003 . A relatively slow change of concept is the representation of concept drift, and an abrupt change in concept is the representation of concept shift BIB003 . The general focus of machine learning is the representation of one type of concept drift. There are many proposed methods of Hoeffding Trees for data streams. Most of these are built on characteristics for dealing with distribution in data streams such as concept drift. These models can be a single Hoeffding Tree, decision trees based on the Hoeffding bound, window-based Hoeffding Trees, weight-based Hoeffding Trees, distribution Hoeffding Trees, and/or an ensemble of Hoeffding Trees. Each of these models addresses various attributes of concept drift in data streams or identifying the best approach to classify data in data streams without concept drift. In many instances these models are combined with other techniques to provide improved accuracy, performance, or drift detection.