A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Hybrid Methods <s> We present a fundamentally different approach to classifying traffic flows according to the applications that generate them. In contrast to previous methods, our approach is based on observing and identifying patterns of host behavior at the transport layer. We analyze these patterns at three levels of increasing detail (i) the social, (ii) the functional and (iii) the application level. This multilevel approach of looking at traffic flow is probably the most important contribution of this paper. Furthermore, our approach has two important features. First, it operates in the dark, having (a) no access to packet payload, (b) no knowledge of port numbers and (c) no additional information other than what current flow collectors provide. These restrictions respect privacy, technological and practical constraints. Second, it can be tuned to balance the accuracy of the classification versus the number of successfully classified traffic flows. We demonstrate the effectiveness of our approach on three real traces. Our results show that we are able to classify 80%-90% of the traffic with more than 95% accuracy. <s> BIB001 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Hybrid Methods <s> Several fundamental security mechanisms for restricting access to network resources rely on the ability of a reference monitor to inspect the contents of traffic as it traverses the network. However, with the increasing popularity of cryptographic protocols, the traditional means of inspecting packet contents to enforce security policies is no longer a viable approach as message contents are concealed by encryption. In this paper, we investigate the extent to which common application protocols can be identified using only the features that remain intact after encryption---namely packet size, timing, and direction. We first present what we believe to be the first exploratory look at protocol identification in encrypted tunnels which carry traffic from many TCP connections simultaneously, using only post-encryption observable features. We then explore the problem of protocol identification in individual encrypted TCP connections, using much less data than in other recent approaches. The results of our evaluation show that our classifiers achieve accuracy greater than 90% for several protocols in aggregate traffic, and, for most protocols, greater than 80% when making fine-grained classifications on single connections. Moreover, perhaps most surprisingly, we show that one can even estimate the number of live connections in certain classes of encrypted tunnels to within, on average, better than 20%. <s> BIB002 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Hybrid Methods <s> Intrusion Detection Systems (IDS) are integral components for the detection of malicious code and attacks. Detection methods can be differentiated in signature-based and anomaly-based systems. While the former ones search for well-known patterns which are available in a database, the latter ones build a model of the normal behavior of a network and later on attacks can be detected by measuring significant deviation of the network status against the normal behavior described by the model. Often this requires the availability of the payload of the network packets. If encryption protocols like SSL or SSH are used, searching for attack signatures in the payload is not possible any longer and also the usage of behavior based techniques is limited: Statistical methods like flow evaluation can be used for anomaly detection, but application level attacks hidden in the encrypted traffic can be undetectable. At the moment, only a few systems are designed to cope with encrypted network traffic. Even so, none of these systems can be easily deployed in general because of the need for protocol modifications, special infrastructures or because of high false alarm rates which are not acceptable in a production environment. In this paper, we propose a new IDS for encrypted traffic which identifies command sequences in encrypted network traffic and evaluates the attack possibility of them. The encrypted traffic is clustered and possibilities for different commands are calculated. Based on that, command sequences are analysed. The system evaluates probabilities for commands and command sequences and the likeliness for an attack based on the identified sequences without a decryption of the packets. Because of only using statistical data gathered from the network traffic, the system can be deployed in general. The current prototype of the system focuses on the command evaluation. <s> BIB003
Karagiannis et al. BIB001 focused on host-based classification. Their method uses only information from the network level and therefore is not affected by transport layer encryption (e.g. TLS). The authors classified the behaviour of the hosts on social, functional and application levels, without access to packet payloads or headers. Each level was classified independently and a cross-level classification was performed afterwards. Particular applications were represented using graphlets, see Figure 7 , which are representations of the application's behaviour. The authors then used heuristics to refine the classification. The ground truth for captured data sets was established by using a signature-based payload classification. Without using transport layer information, only the following traffic classes were identified: web, p2p, data (FTP, database), network management (DNS, SNMP, NTP), mail (SMTP, POP, IMAP), news (NNTP), chat (IRC, AIM, MSN messenger), streaming and gaming. Wright et al. BIB002 worked on a classification of traffic in encrypted tunnels. Multiple flows can be wrapped in a single flow representing the encrypted tunnel. The information from the packet headers was not applicable, therefore the authors used only packet sizes, timing and communication direction. A k-nearest neighbour classifier was used for classification when all TCP connections in a set carried the same application protocol. When TCP connections carried different application protocols, the authors used Hidden Markov Models. The authors also demonstrated that it is possible to determine the number of flows in an encrypted tunnel. The port numbers were used to obtain a ground truth for the captured data set. The authors argue that mislabelled data only decreased the efficiency of their classification algorithm and therefore the real accuracy would be even higher than the reported one. The classifiers were able to detect the following application protocols: HTTP, HTTPS, SMTP, AIM, FTP, SSH, Telnet. Koch and Rodosek BIB003 proposed a system for detecting interactive attacks using SSH. Packet sizes, IP addresses and packet inter-arrival times were used to create clusters of packets which were likely to match a SSH command and its corresponding response. The SSH protocol was recognized based on the port number, and individual commands were identified from the clusters. Following this, sequences of commands were evaluated and possible malicious sequences were reported. The system allows for the customization of malicious sequences' definitions using a subgoals characterization. Each sub-goal maps to a malicious event, such as data gathering or system manipulation. The results from the evaluation of the proposed method show that such identification is possible. Khakpour and Liu  used an entropy of packet payloads for classifying traffic. The authors showed how to compute the entropy of files and how to modify the formula for on-the-fly computation. Several entropy values were computed for each packet. CART and support vector machine (SVM) methods were used for a subsequent classification based on the computed values. Their results demonstrated that SVM methods provide comparable accuracy with less false positives. The authors argue that it is necessary to exclude application layer headers such as HTTP response or picture headers. The reason for this is that computing entropy on the headers leads to a bias and misclassification of a packet. Therefore, a cut-off threshold was used to strip application headers from unknown protocols. The traffic was first classified into three categories: text, encrypted or binary. The authors also postulated that the classification can be more fine-grained and they investigated the classification of application protocols. Then, they demonstrated that it is possible to determine an encryption algorithm with a higher accuracy than random guessing, which they found surprising.