A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> With the advent and explosive growth of the global Internet and electronic commerce environments, adaptive/automatic network/service intrusion and anomaly detection in wide area data networks and e-commerce infrastructures is fast gaining critical research and practical importance. We present and demonstrate the use of a general-purpose hierarchical multitier multiwindow statistical anomaly detection technology and system that operates automatically, adaptively, and proactively, and can be applied to various networking technologies, including both wired and wireless ad hoc networks. Our method uses statistical models and multivariate classifiers to detect anomalous network conditions. Some numerical results are also presented that demonstrate that our proposed methodology can reliably detect attacks with traffic anomaly intensity as low as 3-5 percent of the typical background traffic intensity, thus promising to generate an effective early warning. <s> BIB001 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> This paper presents a new approach inspired by immunology for system performance anomaly detection, which combines the negative selection algorithm (NSA) and genetic algorithm, generating a set of fuzzy rules that can characterize the normal and the abnormal. NSA serves as a filter to eliminate invalid detectors and reduce search space. Experiments with synthetic and real data sets are performed to show the applicability of the proposed approach. <s> BIB002 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> For the purpose of anomaly-based intrusion detection in mobile networks, the utilization of profiles, based on hardware signatures, calling patterns, service usage, and mobility patterns, have been explored by various research teams and commercial systems, namely the fraud management system by Hewlett-Packard and Compaq. This paper examines the feasibility of using profiles, which are based on the mobility patterns of mobile users, who make use of public transportation, e.g. bus. More specifically, a novel framework, which makes use of an instance based learning technique, for classification purposes, is presented. In addition, an empirical analysis is conducted in order to assess the impact of two key parameters, the sequence length and precision level, on the false alarm and detection rates. Moreover, a strategy for enhancing the characterization of users is also proposed. Based on simulation results, it is feasible to use mobility profiles for anomaly-based intrusion detection in mobile wireless networks. <s> BIB003 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> While many researches of host anomaly detection system using system calls under UNIX/UNIX-like systems have been done but little in Windows systems, we do the similar research under Windows platforms via tracing the sequences of Windows native APIs which are considered as the Windows system calls. In this article, we first introduce native API briefly and then divide the captured sequences with slide window method to establish normal pattern database. Then support vector machine method is used for anomaly detection due to its advantages in small-scale dataset and generalization capability. The main purpose of this paper is to prove that Windows native APIs are plausibly possible data source for host anomaly detection system under Windows platforms. <s> BIB004 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> In this paper, we investigate the problem of estimating high-density regions from univariate or multivariate data samples. We estimate minimum volume sets, whose probability is specified in advance, known in the literature as density contour clusters. This problem is strongly related to one-class support vector machines (OCSVM). We propose a new method to solve this problem, the one-class neighbor machine (OCNM) and we show its properties. In particular, the OCNM solution asymptotically converges to the exact minimum volume set prespecified. Finally, numerical results illustrating the advantage of the new method are shown. <s> BIB005 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> Malicious intrusion is the behavior that threats a large number of computers; therefore, recent research has focused on devising new techniques to detect and control internet intrusion with high efficiency and low cost. Unfortunately some anomaly detection system (ADS) over machine learning may get some false alarms if the results of machine learning cannot cover all the normal or abnormal data. In this paper, to solve this problem, we introduce a new approach for anomaly detection using contiguous expert voting algorithm (CEVS). At first, we present our framework of the anomaly detection system, and then we define a new algorithm based on data mining, at last we will use this algorithm to detect the internet anomaly and report our experimental result. The results show that the proposed approach can improve the detection performance of the ADS, where traditional anomaly detection system is used. <s> BIB006 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> Security is one of the most important research issues in wireless sensor network (WSN). A Machine Learning (ML) based anomaly detection scheme is proposed, where Bayesian classification algorithm is used to detect anomalous nodes. By the tool NS2, a small number of samples are given and learned, and intrusion detection rules are built, network attack traffic is generated and simulated. And based on this, its detection rate, average detection rate, false positive rate and average false positive rate are evaluated. Experimental results demonstrate that the scheme achieves higher accuracy rate of detection and lower false positive rate than the current important intrusion detection schemes of WSN. <s> BIB007 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> In the fast growing internet commercial transaction base, attacks on Internet infrastructure, anomaly intrusion traffic attacks combined with traditional network intruders, have become one of the most serious threats to the network security. The proposed system of the traffic anomaly detection method is carried out on the principle traces of non intrusive packet header data (statistical wavelet transform) obtained from the internet server traffic basement. Traffic is monitored at regular intervals to obtain a signal that can be analyzed through statistical techniques and compared to historical norms to detect anomalies. The proposed methodology of anomaly intrusion traffic detection envisions statistical non intrusive wavelet transform mechanisms for real-time data source extracted from NetCon server (Internet Service Provider popularly running at Erode Region) over a period of three months at various time intervals. The experimental results suggest that address spoofing by attackers, which imply that such attacks will be invisible to indirect backscatter measurement techniques. Further, at the detailed packet-level characterization (e.g., attack destination ports), there are significant differences between anomaly and traditional intrusion attack measurements. Thus, there is tremendous value in moving towards direct observations to better understand recent intrusion attacks. Traffic Anomaly intrusion measurements additionally provide information inaccessible to traditional network intrusion measurements, enabling us to better understand how to defend against attacks. <s> BIB008 </s> A Review of Anomaly based Intrusion Detection Systems <s> 31 <s> This paper investigates the use of a one-class support vector machine algorithm to detect the onset of system anomalies, and trend output classification probabilities, as a way to monitor the health of a system. In the absence of “unhealthy” (negative class) information, a marginal kernel density estimate of the “healthy” (positive class) distribution is used to construct an estimate of the negative class. The output of the one-class support vector classifier is calibrated to posterior probabilities by fitting a logistic distribution to the support vector predictor model in an effort to manage false alarms. <s> BIB009
Combining multiple independent data sources and studying combined traditional intrusion attack and anomaly intrusion, the anomaly intrusion traffic detection work carried out by M.Thangavel et al BIB008 , provided the statistical wavelet based detection mechanism.The properties such as attack duration, packet count, packet rate, and dominant protocol type match with the two data sets, as is indicated by attack structure. At lean and heavy traffic scenarios, the demand capacity of the server was observed to give better clarity of anomaly intrusion detection although server uptime. Analysis of several traffic anomaly properties which is impossible using traditional intrusion measurements can be performed by a new model that used anomaly intrusion attack measurements. Small businesses seem to be the most common targets of attacks. Traditional measures in understanding and detecting of anomaly intrusion is no more reliable given the current trends of attacking using spoofed address sources. Miao Wang, et al BIB004 , proposed a method suitable for Windows Host Anomaly Detection System, which is used as a supplement for other security mechanisms under windows. It can only detect intrusions which invoke an anomaly sequence by programs. One of the general situations such as an unauthenticated use of normal programs cannot be detected. Constantine Manikopoulos et al BIB001 , proposed a statistical anomaly detection technology called HIDE with hierarchical multitier multi-observation window system to monitor network traffic parameters simultaneously, using a real-time probability distribution function(PDF) for each parameter, collected during the observation window. The similarity measurements of measured PDF and reference PDF are combined into an anomaly status vector classified by a neural network. This methodology detects attacks and soft faults with traffic anomaly intensity as low as 3 to 5 percent of typical background traffic intensity, thereby generating an early warning. Jeyanthi Hall et al  , proposed an anomaly based intrusion detection system for mobile networks, based on simulation results of mobility profiles for enhancing ABID in mobile wireless networks. If the mobility behavior of users has not been accurately found, the selection of specific values for key parameters, such as sequence length and cluster size is absurd. One possible strategy for enhancing the characterization of users and dealing with concept drift (keeping UMP up to date), is to maintain a window of the newly observed sequences (analogous to the exponential weighted moving average) that can then be used to update the training patterns periodically and hence reduce the false positives. An intrusion detection algorithm and its architecture (twolayered, global central layer and a local layer, together performing data collection, analysis and response), based on data mining and useful in real time for network security, is proposed be HAZEM M. EL-BAKRY et al  .By filtering out the known traffic behavior (intrusive and normal) this IDS focuses on analysis on unknown data thereby reducing false alarm rates. Tich Phuoc Tran et al  proposed an approach called "A Multi-Expert Classification Framework with Transferable Voting for Intrusion Detection". This model, aimed to improve the strategies to detect different anomalies and intrusions emphasized on different attribute selection strategies, defined a new multi expert classification system to test the limits in accuracy and robustness in existing systems and showed that some learning algorithms that use certain set of features can provide superior detection capability for a given attack category. A set of five local classifiers, created to detect five different classes including Normal, Probe, DoS, U2R and R2L and outputs from these experts are then integrated by different voting methods. Finally concluded that a) the weighted voting strategies outperform simple majority voting, b) with Transferable voting approach, the model achieved noticeable performance improvement compared with other conventional techniques, in terms of detection accuracy and system robustness, misclassification cost and processing overheads for "unknown" instances. This may not be an ultimate choice for security, but is effective on different situations. Min Yang et al BIB006 discussed a model based on contiguous expert voting algorithm. Although early methods detect most anomalies, unsuccessful match doesn"t mean an abnormity, as normal rules may not cover all normal data. Detection rate in this is not commendable but it has vast future scope for improvement. The One Class Neighbor Machine (OCNM) algorithm proposed by Munoz and Moguerza, assumes a sample set S comprising T, F-dimensional data points, {x t } T t=1 and estimates minimum volume sets BIB005 . The algorithm requires the choice of a sparsity measure( like choices of a sparsity measure are the k th nearest neighbor Euclidean distance and the average of the first k nearest-neighbor distances) denoted by g, whose values are sorted by a set of points S. The points that lie inner to the minimum volume set (MVS) with minimum sparsity measure g, till a specified number of points μ are obtained. The distance from every point x t to every other point in the sample set has to be found out along with OCNM algorithm ,when the sparsity measure used is kth neighbor distance function. The complexity her is of the order O(T 2 F) as every point is F-dimensional. BIB009 is found as interesting machine learning model for anomaly detection. Use of a SVM with one-class to detect the system anomalies at their early stage is studied along with drift output classification probabilities. Experimentally, absence of failure training data under one-class SVM leads to quick detection of unknown anomalies. Initially dividing the training data into multiple unrelated lower dimensional models, the test data will be evaluated on each model separately thereby revealing outliers in different capacities (as is used to evaluate the posterior class probabilities in Bayesian framework). A Machine Learning (ML) based anomaly detection scheme is proposed by Zhenghong Xiao et al BIB007 , where Bayesian classification algorithm is used to detect anomalous nodes with respect to the characteristics of Wireless Sensor Networks (WSN). While the traditional models failure is attributed to the use of specific characteristics of WSN, this scheme achieves relatively highly accurate detection rate and lower false positive rate The WSN characteristics, such as limited power, limited communication capacity, and limited computational capabilities of nodes are retained in the current model. Establishment of detection rules is also possible after the completion of few samples learning.By making use of the simulation model the authors of the paper tried to provide a proof that the model proposed will be able to retrieve vital association rules and can be able to differentiate normal connection, called intrusion and the intrusion which is unknown with high degree of accuracy. Li-li Liu et al  proposed a method which is used for detecting anomalies which used the construction of a Wavelet Neural Network(WNN) that makes use of changed quantumbehaved particle swarm optimization(MQPSO) algorithm. In this model, MQPSO will be used to train WNN. The learning algorithm multidimensional vector that has WNN parameters as components will be considered as particle. The vector of parameters will search the value globally. Experiments were performed by the authors on well-known KDD Cup 1999 Intrusion Detection Data Set. A result was published by the author that explains that the learning algorithm has more rapid convergence, improved global convergence ability when compared with the traditional quantum-behaved particle swarm optimization (QPSO), thus enhancing the accuracy of anomaly detection. Jian Xu et al BIB002 , inspired by immunology, proposed an approach for detecting system performance anomalies that combines the negative selection algorithm(NSA) and the genetic algorithm. This generated B set of fuzzy rules that will characterize the normal and the abnormal. To remove the invalid detections and to reduce the space required for search, NSA serves as filter. The problem of anomaly detection was derived as two-class classification problem by the authors, the goal being classifying the patterns of the system to two categories as self and non-self, that uses a set of samples which include self and non-self. The idea was to make use of the fuzzy rules in the place of crisp rules. Various concepts of fuzzy logic are used to find a solution to this problem of classification. Taking a set of samples, that include self and non-self, fuzzy detector rules must be generated in the nonself space that will help find a new sample belongs to which category: self or non-self. Making use of the fuzzy rules, the accuracy of detection is improved and these rules, like any other approach do not use hyper-rectangles for representation. Following are the advantages: Many times, the negative selection algorithm acts as a filter. It is added to the Genetic Algorithm (GA) particularly as a filter operator that efficiently eliminates the bad solutions to get low False Positives (FP). A new partial match of chromosome, used to calculate the distance of two different individuals is adopted by the algorithm. A better characterization of the boundaries for the two categories self and non-self is provided by fuzzy rules and they help in reducing the search space. An adaptive approach to modeling user's behavior in computer anomaly detection systems was proposed by Artem M.Sokolov et al  .Markov chains with variable memory length were used as a base model. Considering the changes in user"s behaviour an adaptive version of the algorithm constructing a model was introduced. Based on the demonstrations from the experimental results it was claimed that the modified model can be applied in modeling various sequences. It was possible for the approach to capture more subtle peculiarities of users' behavior and to continuously refine their models only with use of adaptive adjustment procedure. Since the anomaly criteria cannot be precisely formalized, it is a better option to use a set of techniques from which the conclusion will be drawn on the presence of intrusion. Threshold classification and cross test technique are the two possible techniques of anomaly detection, proposed in this work along with their experimental testing. Based on the mobility patterns of mobile users, who make use of public transportation Jeyanthi Hall et al BIB003 examined the feasibility of using profiles. A novel framework, which makes use of an instance based learning technique, for classification purposes was presented by them It was concluded by the authors that it is feasible to use mobility profiles for Anomaly Based Intrusion Detection in mobile wireless networks, based on the simulation results. It is a challenge to accurately characterize the mobility behavior of users. To incorporate the missing parameter sequences into the training patterns, which enhances the characterization of users and increases the detection rate at a minimal cost (low percentage of FAs) is one of the simple strategies. Concept drift, that accommodates variability in mobility sequences over time can also be addressed by continuously monitoring the false alarm rate and incorporating the observed mobility sequences into the training patterns selectively, using a window trhat will be shifted in time. The basis for the selection can be pre-established thresholds, like the frequency of all new sequences encountered over a time period. An approach for generating the data was proposed by Ramkumar Chinchani et al  that is based on customizable templates, where a user profile is represented by each template. The templates can be user-defined or can be created from known data sets. The tools that have been developed is RACOON, that generated generates large amounts of user command data rapidly from a given template. Several statistical similarity tests were performed to test their reliability and to explain that this technique will produce realistic data. To generate the user command data, two paths are there, in the first path, a template specified by the user will be created and provided as the input to the module that generates data. To assist the user, the authors have implemented a front end because the manual specification can be a onerous process. An available data set can be processed for creating the template and the second process, and later it follows the first path. Parameters like data size which are controlled by the user allow the data generation that is of the required size and quality. A new Non-negative Matrix Factorization (NMF) based model to profile program and user behaviors for anomaly intrusion detection was proposed by Wei Wang, et al [50] . The information source in this method is the audit data stream obtained from sequences of system calls and commands. The audit data is divided into segments of fixed length. The frequencies of individual system calls or commands embedded in each segment of the data are the measures for program and user behaviors and are to extract the features from the blocks of audit data associated with the normal behaviors, NMF is used. Based on these features the model describing the normal program and user behaviors is built, deviation from the normal program and user behaviors above a predetermined threshold is considered as anomalous. Unlike many other methods which use transition property, this method considers the frequency property of system calls and commands. It is proven by the simulation results published by the authors that there is no need to consider individual system calls or commands, thus it can be concluded that the computational cost of NMF method is economical and suits real-time intrusion detection.