We describe a novel approach to monitoring high level behaviors using concepts from AI planning. Our goal is to understand what a program is doing based on its system call trace. This ability is particularly important for detecting malware. We approach this problem by building an abstract model of the operating system using the STRIPS planning language, casting system calls as planning operators. Given a system call trace, we simulate the corresponding operators on our model and by observing the properties of the state reached, we learn about the nature of the original program and its behavior. Thus, unlike most statistical detection methods that focus on syntactic features, our approach is semantic in nature. Therefore, it is more robust against obfuscation techniques used by malware that change the outward appearance of the trace but not its effect. We demonstrate the efficacy of our approach by evaluating it on actual system call traces.
Initially, only system call indices were used as features @cite_0 @cite_13 . @cite_7 compares three different representations of system calls: @math -grams of system call names, histograms of system call names, and individual system calls with associated parameters. @cite_6 proposes the use of system call sequences of varying length as features. @cite_7 @cite_15 investigate extracting features for machine learning from arguments of system calls. @cite_17 studies novel techniques of anomaly detection and classification using @math -grams of system calls. @cite_20 conducts a case study of @math -gram based feature selection for system-call based monitoring, and analyses the influence of the size of the @math -gram set and the maximum @math -gram length on detection accuracy.
@cite_7 @cite_6 @cite_0 @cite_15 @cite_13 @cite_20 @cite_17
2138209477 1984350393 2145195811 2143677609
This paper investigates anomaly detection techniques that have been successful for detecting external threats and applies them to the insider threat problem. The "insider threat" involves the actions of a trusted and privileged user who is inappropriately accessing or disseminating sensitive information or otherwise compromising information systems. In contrast, the "external threat" involves the actions of an outsider attempting to compromise or gain access to the information systems. Although approaches for automatically detecting external threat instances have been quite successful (i.e., intrusion detection systems), there is very little similar work for the insider threat. In the past, anomaly detection systems have proven useful for detecting external threat. Anomaly detection at the system call level offers a high degree of information assurance in terms of tamper-resistance and system activity coverage. Therefore, we investigate three system-call-based feature representations: n-grams of system call names, histograms of system call names, and individual system calls with associated parameters. We find that none of these representations consistently performs as well when dealing with the internal threat as previous results show for external threat detection. However, parameter-based features for certain system calls do show some sensitivity to detecting the insider threat, and we plan to explore and enhance this sensitivity in future work. Intrusion detection systems (IDSs) are used to detect traces of malicious activities targeted against the network and its resources. Anomaly-based IDSs build models of the expected behavior of applications by analyzing events that are generated during the applications' normal operation. Once these models have been established, subsequent events are analyzed to identify deviations, on the assumption that anomalies represent evidence of an attack. Host-based anomaly detection systems often rely on system call sequences to characterize the normal behavior of applications. Recently, it has been shown how these systems can be evaded by launching attacks that execute legitimate system call sequences. The evasion is possible because existing techniques do not take into account all available features of system calls. In particular, system call arguments are not considered. We propose two primary improvements upon existing host-based anomaly detectors. First, we apply multiple detection models to system call arguments. Multiple models allow the arguments of each system call invocation to be evaluated from several different perspectives. Second, we introduce a sophisticated method of combining the anomaly scores from each model into an overall aggregate score. The combined anomaly score determines whether an event is part of an attack. Individual anomaly scores are often contradicting and, therefore, a simple weighted sum cannot deliver reliable results. To address this problem, we propose a technique that uses Bayesian networks to perform system call classification. We show that the analysis of system call arguments and the use of Bayesian classification improves detection accuracy and resilience against evasion attempts. In addition, the paper describes a tool based on our approach and provides a quantitative evaluation of its performance in terms of both detection effectiveness and overhead. A comparison with four related approaches is also presented. This paper discusses our research in developing a generalized and systematic method for anomaly detection. The key ideas are to represent normal program behaviour using system call frequencies and to incorporate probabilistic techniques for classification to detect anomalies and intrusions. Using experiments on the sendmail system call data, we demonstrate that concise and accurate classifiers can be constructed to detect anomalies. An overview of the approach that we have implemented is provided. For the past 30+ years, system calls have been the de facto interface used by applications to request services from the operating system kernel. System calls have almost universally been implemented as a synchronous mechanism, where a special processor instruction is used to yield userspace execution to the kernel. In the first part of this paper, we evaluate the performance impact of traditional synchronous system calls on system intensive workloads. We show that synchronous system calls negatively affect performance in a significant way, primarily because of pipeline flushing and pollution of key processor structures (e.g., TLB, data and instruction caches, etc.). We propose a new mechanism for applications to request services from the operating system kernel: exception-less system calls. They improve processor efficiency by enabling flexibility in the scheduling of operating system work, which in turn can lead to significantly increased temporal and spacial locality of execution in both user and kernel space, thus reducing pollution effects on processor structures. Exception-less system calls are particularly effective on multicore processors. They primarily target highly threaded server applications, such as Web servers and database servers. We present FlexSC, an implementation of exceptionless system calls in the Linux kernel, and an accompanying user-mode thread package (FlexSC-Threads), binary compatible with POSIX threads, that translates legacy synchronous system calls into exception-less ones transparently to applications. We show how FlexSC improves performance of Apache by up to 116 , MySQL by up to 40 , and BIND by up to 105 while requiring no modifications to the applications.