A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> We present a fundamentally different approach to classifying traffic flows according to the applications that generate them. In contrast to previous methods, our approach is based on observing and identifying patterns of host behavior at the transport layer. We analyze these patterns at three levels of increasing detail (i) the social, (ii) the functional and (iii) the application level. This multilevel approach of looking at traffic flow is probably the most important contribution of this paper. Furthermore, our approach has two important features. First, it operates in the dark, having (a) no access to packet payload, (b) no knowledge of port numbers and (c) no additional information other than what current flow collectors provide. These restrictions respect privacy, technological and practical constraints. Second, it can be tuned to balance the accuracy of the classification versus the number of successfully classified traffic flows. We demonstrate the effectiveness of our approach on three real traces. Our results show that we are able to classify 80%-90% of the traffic with more than 95% accuracy. <s> BIB001 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> Most tools to recognize the application associated with network connections use well-known signatures as basis for their classification. This approach is very effective in enterprise and campus networks to pinpoint forbidden applications (peer to peer, for instance) or security threats. However, it is easy to use encryption to evade these mechanisms. In particular, Secure Sockets Layer (SSL) libraries such as OpenSSL are widely available and can easily be used to encrypt any type of traffic. In this paper, we propose a method to detect applications in SSL encrypted connections. Our method uses only the size of the first few packets of an SSL connection to recognize the application, which enables an early classification. We test our method on packet traces collected on two campus networks and on manually-encrypted traces. Our results show that we are able to recognize the application in an SSL connection with more than 85% accuracy. <s> BIB002 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> The identification of application flows is a critical task in order to manage bandwidth requirements of different kind of services (i.e. VOIP, Video, ERP). As network security functions spread, an increasing amount of traffic is natively encrypted due to privacy issues (e.g. VPN). This makes ineffective current traffic classification systems based on ports and payload inspection, e.g. even powerful Deep Packet Inspection is useless to classify application flow carried inside SSH sessions. We have developed a real time traffic classification method based on cluster analysis to identify SSH flows from statistical behavior of IP traffic parameters, such as length, arrival times and direction of packets. In this paper we describe our approach and relevant obtained results. We achieve detection rate up to 99.5 % in classifying SSH flows and accuracy up to 99.88 % for application flows carried within those flows, such as SCP, SFTP and HTTP over SSH. <s> BIB003 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> The increasing use of encrypted traffic combined with non-standard port associations makes the task of traffic identification increasingly difficult. This work adopts a multi-objective clustering approach to the problem in which a Genetic Algorithm performs both feature selection and cluster count optimization under a flow based representation. Solutions do not use port numbers, IP address or payload. Performance of the resulting model provides 90% detection 0.8% false positive rates with 13 clusters supported by 14 of the original 38 features. <s> BIB004 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> Classifying network flows by their application type is the backbone of many crucial network monitoring and controlling tasks, including billing, quality of service, security and trend analyzers. The classical “port-based” and “payload-based” approaches to traffic classification have several shortcomings. These limitations have motivated the study of classification techniques that build on the foundations of learning theory and statistics. The current paper presents a new statistical classifier that allows real time classification of encrypted data. Our method is based on a hybrid combination of the k-means and k-nearest neighbor (or k-NN) geometrical classifiers. The proposed classifier is both fast and accurate, as implied by our feasibility tests, which included implementing and intergrading statistical classification into a realtime embedded environment. The experimental results indicate that our classifier is extremely robust to encryption. <s> BIB005 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> An important part of network management requires the accurate identification and classification of network traffic for decisions regarding bandwidth management, quality of service, and security. This work explores the use of a Multi-Objective Genetic Algorithm (MOGA) for both, feature selection and cluster count optimization, for an unsupervised machine learning technique, K-Means, applied to encrypted traffic identification. Specifically, a hierarchical K-Means algorithm is employed, comparing its performance to the MOGA with a non-hierarchical (flat) K-Means algorithm. The latter has already been benchmarked against common unsupervised techniques found in the literature, where results have favored the proposed MOGA. The purpose of this paper is to explore the gains, if any, obtained by increasing cluster purity in the proposed model by means of a second layer of clusters. In this work, SSH is chosen as an example of an encrypted application. However, nothing prevents the proposed model to work with other types of encrypted traffic, such as SSL or Skype. Results show that with the hierarchical MOGA, significant gains are observed in terms of the classification performance of the system. <s> BIB006 </s> A Survey of Methods for Encrypted Traffic Classification andAnalysis <s> Semi-Supervised Machine Learning Methods <s> Classification analysis of network traffic based on port number or payload is becoming increasingly difficult from security to quality of service measurements, because of using dynamic port numbers, masquerading and various cryptographic techniques to avoid detection. Research tends to analyze flow statistical features with machine learning techniques. Clustering approaches do not require complex training procedure and large memory cost. However, the performance of clustering algorithm like k-Means still have own disadvantages. We propose a novel approach of considering harmonic mean as distance matric, and evaluate it in terms of three metrics on real-world encrypted traffic. The result shows the classification has better performance compared with the previously. <s> BIB007
Bernaille and Teixeira BIB002 used traffic clustering to detect applications encrypted by SSL. Their method has three steps. First, they detected SSL connections using a clustering algorithm (Gaussian Mixture Model) on packet sizes and directions of initial packets of a connection. The first three packets and 35 clusters provide good accuracy in detecting SSL traffic. After the SSL traffic is identified, the first data packets of the connections are identified. The sizes of the data packets are used by a clustering algorithm to detect an underlying application in the third step. However, the packet sizes are modified in the last step to allow for encryption overhead. The evaluation of the proposed method is done on traffic traces from live networks and a manually generated packet trace. The data sets contain HTTP, POP3, FTP, BitTorrent and eDonkey application protocols encrypted using SSL. The authors also show that using a combination of clustering and port numbers to differentiate between applications in clusters provides better results than clustering alone. Maolini et al. BIB003 identify SSH traffic and determine underlying protocols (SCP, SFTP, HTTP) using a k-means algorithm. Only three packet features are used: the direction, the number of bytes and the timestamps of each packet. Their private data sets are created from artificial traffic and contain HTTP, FTP, POP3 and SSH protocols. Control packets such as TCP handshake, retransmitted packets and ACK only packets are removed from statistics as they negatively affect the precision. Authors use only first 3-7 packets to achieve a real-time identification. Backquet et al. BIB004 BIB006 use a Multi-Objective Genetic Algorithm to select a flow feature subspace and parameters for a clustering algorithm which detects encrypted traffic. The second work employs a hierarchical k-means algorithm to increase the identification's accuracy. The authors evaluate both approaches on a private data set captured at a university campus. SSH is used as a representative of encrypted traffic and the ground truth is gained from the payload of the captured packets. Based on previous works, the authors argue that the feature selection and number of clusters highly affect the overall accuracy. Therefore, the authors selected four objectives for the genetic algorithm: cluster cohesiveness, cluster separation, the number of clusters and the amount of used flow features. The results show (a) that only 14 from a total of 38 flow features were used by the best-performing algorithm and (b) that using a hierarchical k-means algorithm increases the identification performance. Bar-Yanai et al. BIB005 combined k-means and k-nearest neighbour clustering algorithms to construct a new, real-time classifier for encrypted traffic. The resulting classification algorithm has the light weight complexity of the k-means algorithm and accuracy of the k-nearest neighbour algorithm. They claim the method is fast, accurate and robust in regard to encryption, asymmetric routing and packet ordering. A labelled data set was prepared from generated samples of the traffic and the classification of the data set was done using payloads of the packets. Flows shorter than 15 packets were removed from the data set, since real-time classification for such short flows is not of practical use. If available, the first 100 packets were used for the classification. The authors stress that their method is applicable in a real-time environment and tested their implementation on an ISP link. The application protocols classified are HTTP, SMTP, POP3, Skype, eDonkey, BitTorrent, Encrypted BitTorrent, RTP and ICQ. Zhang et al. BIB007 propose an improvement to the k-means clustering algorithm. Using harmonic mean to reduce the impact of random initial clustering centres, the authors are able to increase Figure 7 . A visual representation of transport-layer interactions for various applications BIB001 . applications such as Twitter, Skype and Dropbox. The Markov chain fingerprints are based on protocol specific distributions of packets in time. Data sets were captured from real traffic, contain only SSL/TLS traffic and were not published. The ground truth was obtained by inspecting domain names of the SSL/TLS traffic. The authors discovered that many protocol implementations differ from the RFC specification, which required them to adjust the fingerprints.