As the size and source of network traffic increase, so does the challenge of monitoring and analyzing network traffic. Therefore, sampling algorithms are often used to alleviate these scalability issues. However, the use of high entropy data streams, through the use of either encryption or compression, further compounds the challenge as current state-of-the-art algorithms cannot accurately and efficiently differentiate between encrypted and compressed packets. In this paper, we propose a novel traffic classification method named High Entropy DistinGuishEr (HEDGE) to distinguish between compressed and encrypted traffic. HEDGE is based on the evaluation of the randomness of the data streams and can be applied to individual packets without the need to have access to the entire stream. The findings from the evaluation show that our approach outperforms current state of the art. We also make available our statistically sound dataset, based on known benchmarks, to the wider research community.
In network traffic analysis approaches, data streams are analysed and classified according to flow and packet information, payload content and statistics (including among others packet size and average times). Based on such information, we can distinguish between: (i) techniques, which determine a set of characteristics features regarding the traffic flow as well as internal packet information and its structure (i.e. it differs depending on the protocol used), and (ii) techniques, which inspect the data payload content and apply relevant methods to extract statistical properties and identify, for example, the type of information being transmitted (e.g. text, image, video, music, compressed or encrypted). We refer the interested reader to @cite_22 @cite_18 for a classification of network traffic analysis models based on input data, applied techniques applied and their corresponding outcomes.
@cite_18 @cite_22
2073089243 2140674594 2164210932 2468337398
The identification of network applications through observation of associated packet traffic flows is vital to the areas of network management and surveillance. Currently popular methods such as port number and payload-based identification exhibit a number of shortfalls. An alternative is to use machine learning (ML) techniques and identify network applications based on per-flow statistics, derived from payload-independent features such as packet length and inter-arrival time distributions. The performance impact of feature set reduction, using Consistency-based and Correlation-based feature selection, is demonstrated on Naive Bayes, C4.5, Bayesian Network and Naive Bayes Tree algorithms. We then show that it is useful to differentiate algorithms based on computational performance rather than classification accuracy alone, as although classification accuracy between the algorithms is similar, computational performance can differ significantly. Identifying applications and classifying network traffic flows according to their source applications are critical for a broad range of network activities. Such a decision can be based on packet header fields, packet payload content, statistical characteristics of traffic and communication patterns of network hosts. However, most present techniques rely on some sort of a priori knowledge, which means they require labor-intensive preprocessing before running and cannot deal with previously unknown applications. In this paper, we propose a traffic classification system based on application signatures, with a novel approach to fully automate the process of deriving signatures from unidentified traffic. The key idea is to integrate statistics-based flow clustering with payload-based signature matching method, so as to eliminate the requirement of pre-labeled training data sets. We evaluate the efficiency of our approach using real-world traffic trace, and the results indicate that signature classifiers built from clustered data and pre-labeled data are able to achieve similar high accuracy better than 99p. Copyright © 2010 John Wiley & Sons, Ltd. The increasing practicality of large-scale flow capture makes it possible to conceive of traffic analysis methods that detect and identify a large and diverse set of anomalies. However the challenge of effectively analyzing this massive data source for anomaly diagnosis is as yet unmet. We argue that the distributions of packet features (IP addresses and ports) observed in flow traces reveals both the presence and the structure of a wide range of anomalies. Using entropy as a summarization tool, we show that the analysis of feature distributions leads to significant advances on two fronts: (1) it enables highly sensitive detection of a wide range of anomalies, augmenting detections by volume-based methods, and (2) it enables automatic classification of anomalies via unsupervised learning. We show that using feature distributions, anomalies naturally fall into distinct and meaningful clusters. These clusters can be used to automatically classify anomalies and to uncover new anomaly types. We validate our claims on data from two backbone networks (Abilene and Geant) and conclude that feature distributions show promise as a key element of a fairly general network anomaly diagnosis framework. With the widespread use of encrypted data transport, network traffic encryption is becoming a standard nowadays. This presents a challenge for traffic measurement, especially for analysis and anomaly detection methods, which are dependent on the type of network traffic. In this paper, we survey existing approaches for classification and analysis of encrypted traffic. First, we describe the most widespread encryption protocols used throughout the Internet. We show that the initiation of an encrypted connection and the protocol structure give away much information for encrypted traffic classification and analysis. Then, we survey payload and feature-based classification methods for encrypted traffic and categorize them using an established taxonomy. The advantage of some of described classification methods is the ability to recognize the encrypted application protocol in addition to the encryption protocol. Finally, we make a comprehensive comparison of the surveyed feature-based classification methods and present their weaknesses and strengths. Copyright © 2015 John Wiley & Sons, Ltd.