15943138
Constructing attribute weights from computer audit data for effective intrusion detection
Attributes construction and selection from audit data is the first and very important step for anomaly intrusion detection. In this paper, we present several cross frequency attribute weights to model user and program behaviors for anomaly intrusion detection. The frequency attribute weights include plain term frequency (TF) and various forms of term frequency-inverse document frequency (tfidf), referred to as Ltfidf, Mtfidf and LOGtfidf. Nearest Neighbor (NN) and k-NN methods with Euclidean and Cosine distance measures as well as principal component analysis (PCA) and Chi-square test method based on these frequency attribute weights are used for anomaly detection. Extensive experiments are performed based on command data from Schonlau et al. The testing results show that the LOGtfidf weight gives better detection performance compared with plain frequency and other types of weights. By using the LOGtfidf weight, the simple NN method and PCA method achieve the better masquerade detection results than the other 7 methods in the literature while the Chi-square test consistently returns the worst results. The PCA method is suitable for fast intrusion detection because of its capability of reducing data dimensionality while NN and k-NN methods are suitable for detection of a small data set because of its no need of training process. A HTTP log data set collected in a real environment and the sendmail system call data from University of New Mexico (UNM) are used as well and the results also demonstrate the effectiveness of the LOGtfidf weight for anomaly intrusion detection.
13221631
Fast intrusion detection based on a non-negative matrix factorization model
In this paper, we present an efficient fast anomaly intrusion detection model incorporating a large amount of data from various data sources. A novel method based on non-negative matrix factorization (NMF) is presented to profile program and user behaviors of a computer system. A large amount of high-dimensional data is collected in our experiments and divided into smaller data blocks by a specific scheme. The system call data is divided into blocks by processes, while command data is divided into consecutive blocks with a fixed length. The frequencies of individual elements in each block of data are computed and placed column by column as data vectors to construct a matrix representation. NMF is employed to reduce the high-dimensional data vectors and anomaly detection can be realized as a very simple classifier in low dimensions. Experimental results show that the model presented in this paper is promising in terms of detection accuracy, computation efficiency and implementation for fast intrusion detection.
15381526
A general framework for adaptive and online detection of web attacks
Detection of web attacks is an important issue in current defense-in-depth security framework. In this paper, we propose a novel general framework for adaptive and online detection of web attacks. The general framework can be based on any online clustering methods. A detection model based on the framework is able to learn online and deal with "concept drift" in web audit data streams. Str-DBSCAN that we extended DBSCAN to streaming data as well as StrAP are both used to validate the framework. The detection model based on the framework automatically labels the web audit data and adapts to normal behavior changes while identifies attacks through dynamical clustering of the streaming data. A very large size of real HTTP Log data collected in our institute is used to validate the framework and the model. The preliminary testing results demonstrated its effectiveness.
2256492
Autonomic Intrusion Detection System
We propose a novel framework of autonomic intrusion detection that fulfills online and adaptive intrusion detection in unlabeled audit data streams. The framework owns ability of self-managing: self-labeling, self-updating and self-adapting. Affinity Propagation (AP) uses the framework to learn a subject's behavior through dynamical clustering of the streaming data. The testing results with a large real HTTP log stream demonstrate the effectiveness and efficiency of the method.
206579866
Efficient detection of DDoS attacks with important attributes
DDoS attacks are major threats in current computer networks. However, DDoS attacks are difficult to be quickly detected. In this paper, we introduce a system that only extracts several important attributes from network traffic for DDoS attack detection in real computer networks. We collect a large set of DDoS attack traffic by implementing various DDoS attacks as well as normal data during normal usage. Information Gain and Chi-square methods are used to rank the importance of 41 attributes extracted from the network traffic with our programs. Bayesian networks as well as C4.5 are then employed to detect attacks as well as to determine what size of attributes is appropriate for fast detection. Empirical results show that only using the most important 9 attributes, the detection accuracy remains the same or even has some improvements compared with that of using all the 41 attributes based on Bayesian Networks and C4.5 methods. Only using several attributes also improves the efficiency in terms of attributes constructing, models training as well as intrusion detection.
17218997
Profiling program behavior for anomaly intrusion detection based on the transition and frequency property of computer audit data
Intrusion detection is an important technique in the defense-in-depth network security framework. In recent years, it has been a widely studied topic in computer network security. In this paper, we present two methods, namely, the Hidden Markov Models (HMM) method and the Self Organizing Maps (SOM) method, to profile normal program behavior for anomaly intrusion detection based on computer audit data. The HMM method utilizes the transition property of events while SOM method relies on the frequency property of events. Two data sets, CERT synthetic Sendmail system call data collected in the University of New Mexico (UNM) and Live FTP system call data collected in the CNSIS lab of Xi'an Jiaotong University, were used to assess the two methods. Testing results show that the HMM method using the transition property of events produces good detection performance while high computational expense is required both for training and detection. The HMM method is better than other two methods reported previously in terms of detection accuracy for the same data set. The SOM method considering the frequency property of events, on the other hand, is suitable for real-time intrusion detection because of its capability of processing a large amount of data with low computational overhead.
108674
Sample Selection for Maximal Diversity
The problem of selecting a sample subset sufficient to preserve diversity arises in many applications. One example is in the design of recombinant inbred lines (RIL) for genetic association studies. In this context, genetic diversity is measured by how many alleles are retained in the resulting inbred strains. RIL panels that are derived from more than two parental strains, such as the collaborative cross (Churchill et al., 2004), present a particular challenge with regard to which of the many existing lab mouse strains should be included in the initial breeding funnel in order to maximize allele retention. A similar problem occurs in the study of customer reviews when selecting a subset of products with a maximal diversity in reviews. Diversity in this case implies the presence of a set of products having both positive and negative ranks for each customer. In this paper, we demonstrate that selecting an optimal diversity subset is an NP-complete problem via reduction to set cover. This reduction is sufficiently tight that greedy approximations to the set cover problem directly apply to maximizing diversity. We then suggest a slightly modified subset selection problem in which an initial greedy diversity solution is used to effectively prune an exhaustive search for all diversity subsets bounded from below by a specified coverage threshold. Extensive experiments on real datasets are performed to demonstrate the effectiveness and efficiency of our approach.
17166482
Benchmarking the effectiveness of sequential pattern mining methods
Recently, there is an increasing interest in new intelligent mining methods to find more meaningful and compact results. In intelligent data mining research, accessing the quality and usefulness of the results from different mining methods is essential. However, there is no general benchmarking criteria to evaluate whether these new methods are indeed more effective compared to the traditional methods. Here we propose a novel benchmarking criteria that can systematically evaluate the effectiveness of any sequential pattern mining method under a variety of situations. The benchmark evaluates how well a mining method finds known common patterns in synthetic data. Such an evaluation provides a comprehensive understanding of the resulting patterns generated from any mining method empirically. In this paper, the criteria are applied to conduct a detailed comparison study of the support-based sequential pattern model with an approximate pattern model based on sequence alignment. The study suggests that the alignment model will give a good summary of the sequential data in the form of a set of common patterns in the data. In contrast, the support model generates massive amounts of frequent patterns with much redundancy. This suggests that the results of the support model require more post processing before it can be of actual use in real applications.
22669776
Identities via Bell matrix and Fibonacci matrix
In this paper, we study the relations between the Bell matrix and the Fibonacci matrix, which provide a unified approach to some lower triangular matrices, such as the Stirling matrices of both kinds, the Lah matrix, and the generalized Pascal matrix. To make the results more general, the discussion is also extended to the generalized Fibonacci numbers and the corresponding matrix. Moreover, based on the matrix representations, various identities are derived.
38794991
Localized Recursive Estimation in Wireless Sensor Networks
This paper proposes a localized recursive estimation scheme for parameter estimation in wireless sensor networks. Given any parameter occurred at some location and time, a number of sensors recursively estimates the parameter by using their local measurements of the parameter that is attenuated with the distance between a sensor and the target location and corrupted by noises. Compared with centralized estimation schemes that transmit all measurements to a sink (or a fusion center), the recursive scheme needs only to transmit the final estimate to a sink. When a sink is faraway from the sensors and multihop communications have to be used, using localized recursive estimation can help to reduce energy consumption and reduce network traffic load. Furthermore, the most efficient sequence of sensors for estimation is defined and the necessary condition for such a sequence is determined. Some numerical examples are also provided. By using some typical industrial sensor parameter values, it is shown that recursive scheme consumes much less energy when the sink is three hops or more faraway from the local sensors.
11942108
Subspace Partitioning for Target Detection and Identification
Detection of a given target or set of targets from observed data is a problem countered in many applications. Regardless of the algorithm selected, detection performance can be severely degraded when the subspace defined by the target data set is singular or ill conditioned. High correlations between target components and their linear combinations lead to false positives and misidentifications, especially for subspace-based detectors. In this paper, we propose a subspace partitioning scheme that allows for detection to be performed in a number of better conditioned subspaces instead of the original subspace. The proposed technique is applied to Raman spectroscopic data analysis. Through both simulation and experimental results, we demonstrate the improvement in the overall detection performance when using the proposed subspace partitioning scheme in conjunction with several subspace detection methods that are commonly used in practice.