On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> Today's Internet intrusion detection systems (IDSes) monitor edge networks' DMZs to identify and/or filter malicious flows. While an IDS helps protect the hosts on its local edge network from compromise and denial of service, it cannot alone effectively intervene to halt and reverse the spreading of novel Internet worms. Generation of the worm signatures required by an IDS--the byte patterns sought in monitored traffic to identify worms--today entails non-trivial human labor, and thus significant delay: as network operators detect anomalous behavior, they communicate with one another and manually study packet traces to produce a worm signature. Yet intervention must occur early in an epidemic to halt a worm's spread. In this paper, we describe Autograph, a system that automatically generates signatures for novel Internet worms that propagate using TCP transport. Autograph generates signatures by analyzing the prevalence of portions of flow payloads, and thus uses no knowledge of protocol semantics above the TCP level. It is designed to produce signatures that exhibit high sensitivity (high true positives) and high specificity (low false positives); our evaluation of the system on real DMZ traces validates that it achieves these goals. We extend Autograph to share port scan reports among distributed monitor instances, and using trace-driven simulation, demonstrate the value of this technique in speeding the generation of signatures for novel worms. Our results elucidate the fundamental trade-off between early generation of signatures for novel worms and the specificity of these generated signatures. <s> BIB001 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> We present a new kind of network perimeter monitoring strategy, which focuses on recognizing the infection and coordination dialog that occurs during a successful malware infection. BotHunter is an application designed to track the two-way communication flows between internal assets and external entities, developing an evidence trail of data exchanges that match a state-based infection sequence model. BotHunter consists of a correlation engine that is driven by three malware-focused network packet sensors, each charged with detecting specific stages of the malware infection process, including inbound scanning, exploit usage, egg downloading, outbound bot coordination dialog, and outbound attack propagation. The BotHunter correlator then ties together the dialog trail of inbound intrusion alarms with those outbound communication patterns that are highly indicative of successful local host infection. When a sequence of evidence is found to match BotHunter's infection dialog model, a consolidated report is produced to capture all the relevant events and event sources that played a role during the infection process. We refer to this analytical strategy of matching the dialog flows between internal assets and the broader Internet as dialog-based correlation, and contrast this strategy to other intrusion detection and alert correlation methods. We present our experimental results using BotHunter in both virtual and live testing environments, and discuss our Internet release of the BotHunter prototype. BotHunter is made available both for operational use and to help stimulate research in understanding the life cycle of malware infections. <s> BIB002 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> Hosts infected with malicious software, so called malware, are ubiquitous in today's computer networks. The means whereby malware can infiltrate a network are manifold and range from exploiting of software vulnerabilities to tricking a user into executing malicious code. Monitoring and detection of all possible infection vectors is intractable in practice. Hence, we approach the problem of detecting malicious software at a later point when it initiates contact with its maintainer; a process referred to as "phoning home". In particular, we introduce Botzilla, a method for detection of malware communication, which proceeds by repetitively recording network traffic of malware in a controlled environment and generating network signatures from invariant content patterns. Experiments conducted at a large university network demonstrate the ability of Botzilla to accurately identify malware communication in network traffic with very low false-positive rates. <s> BIB003 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> A distinguishing characteristic of bots is their ability to establish a command and control (C&C) channel. The typical approach to build detection models for C&C traffic and to identify C&C endpoints (IP addresses and domains of C&C servers) is to execute a bot in a controlled environment and monitor its outgoing network connections. Using the bot traffic, one can then craft signatures that match C&C connections or blacklist the IP addresses or domains that the packets are sent to. Unfortunately, this process is not as easy as it seems. For example, bots often open a large number of additional connections to legitimate sites (to perform click fraud or query for the current time), and bots can deliberately produce "noise" - bogus connections that make the analysis more difficult. Thus, before one can build a model for C&C traffic or blacklist IP addresses and domains, one first has to pick the C&C connections among all the network traffic that a bot produces. ::: ::: In this paper, we present JACKSTRAWS, a system that accurately identifies C&C connections. To this end, we leverage host-based information that provides insights into which data is sent over each network connection as well as the ways in which a bot processes the information that it receives. More precisely, we associate with each network connection a behavior graph that captures the system calls that lead to this connection, as well as the system calls that operate on data that is returned. By using machine learning techniques and a training set of graphs that are associated with known C&C connections, we automatically extract and generalize graph templates that capture the core of different types of C&C activity. Later, we use these C&C templates to match against behavior graphs produced by other bots. Our results show that JACKSTRAWS can accurately detect C&C connections, even for novel bot families that were not used for template generation. <s> BIB004 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> Abstract. Botmasters increasingly encrypt command-and-control (C&C) communication to evade existing intrusion detection systems. Our detailed C&C traffic analysis shows that at least ten prevalent malware families avoid well-known C&C carrier protocols, such as IRC and HTTP. Six of these families - e.g., Zeus P2P, Pramro, Virut, and Sality - do not exhibit any characteristic n-gram that could serve as payload-based signature in an IDS. ::: ::: Given knowledge of the C&C encryption algorithms, we detect these evasive C&C protocols by decrypting any packet captured on the network. In order to test if the decryption results in messages that stem from malware, we propose ProVex, a system that automatically derives probabilistic vectorized signatures. ProVex learns characteristic values for fields in the C&C protocol by evaluating byte probabilities in C&C input traces used for training. This way, we identify the syntax of C&C messages without the need to manually specify C&C protocol semantics, purely based on network traffic. Our evaluation shows that ProVex can detect all studied malware families, most of which are not detectable with traditional means. Despite its naive approach to decrypt all traffic, we show that ProVex scales up to multiple Gbit/s line speed networks. <s> BIB005 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> The ever-increasing number of malware families and polymorphic variants creates a pressing need for automatic tools to cluster the collected malware into families and generate behavioral signatures for their detection. Among these, network traffic is a powerful behavioral signature and network signatures are widely used by network administrators. In this paper we present FIRMA, a tool that given a large pool of network traffic obtained by executing unlabeled malware binaries, generates a clustering of the malware binaries into families and a set of network signatures for each family. Compared with prior tools, FIRMA produces network signatures for each of the network behaviors of a family, regardless of the type of traffic the malware uses e.g., HTTP, IRC, SMTP, TCP, UDP. We have implemented FIRMA and evaluated it on two recent datasets comprising nearly 16,000 unique malware binaries. Our results show that FIRMA's clustering has very high precision 100% on a labeled dataset and recall 97.7%. We compare FIRMA's signatures with manually generated ones, showing that they are as good often better, while generated in a fraction of the time. <s> BIB006 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Signature-Based Methods <s> Botnets, which are networks of compromised machines under the control of a single malicious entity, are a serious threat to online security. The fact that botnets, by definition, receive their commands from a single entity can be leveraged to fight them. To this end, one requires techniques that can detect command and control (C&C) traffic, as well as the servers that host C&C services. Given the knowledge of a C&C server's IP address, one can use this information to detect all hosts that attempt to contact such a server, and subsequently disinfect, disable, or block the infected machines. This information can also be used by law enforcement to take down the C&C server. In this paper, we present a new botnet C&C signature extraction approach that can be used to find C&C communication in traffic generated by executing malware samples in a dynamic analysis system. This approach works in two steps. First, we extract all frequent strings seen in the network traffic. Second, we use a function that assigns a score to each string. This score represents the likelihood that the string is indicative of C&C traffic. This function allows us to rank strings and focus our attention on those that likely represent good C&C signatures. We apply our technique to almost 2.6 million network connections produced by running more than 1.4 million malware samples. Using our technique, we were able to automatically extract a set of signatures that are able to identify C&C traffic. Furthermore, we compared our signatures with those used by existing tools, such as Snort and BotHunter. <s> BIB007
In signature-based detection methods, malware C&C is detected by looking for known patterns of behavior. Signatures are generated for known malware samples, then new traffic is compared to these signatures BIB004 . Signatures are generated by analyzing confirmed C&C traffic collected from various sources. The primary sources are honeynets and sandboxes. Malware is run in controlled conditions, and its activity is recorded. Most aspects of the malware's network behavior can be included in a signature, from statistical properties such as flow sizes to detailed information such as packet contents. 4.3.1. Communication Pattern Detection. Malware variants often have very particular protocols when it comes to communication. These are often noticeably different compared to legitimate traffic, both in packet contents and in the behavior of the communication. This makes signature-based detection methods very good for detecting known variants of malware. Many different pieces of malware may also be based on a common component, meaning that a single signature can be used to detect multiple pieces of similar malware. A popular method for detection is to produce signatures based on the contents of packets. It is often the case that packets of data involved in the C&C of malware will be almost identical across multiple hosts. Even though some malware families use encryption in their communications, that encryption is usually a simple, lightweight algorithm (as the encryption is often for obscurity rather than security), so there are similarities among different ciphertexts. For example, in the work of BIB003 , n-gram-based signatures are generated for the payloads of malware that is run under controlled conditions in a sandbox. Signatures are also generated for legitimate traffic, and with this method the system can achieve detection rates of close to 95%, with an FP rate of close to zero when running on a network gateway. BIB005 extend this idea by providing a system for handling encrypted C&C channels. They leverage the fact that many malware variants use hard-coded, simple keys, and thus they extract these keys through reverse engineering and attempt to decrypt all packets before pattern matching is applied. BIB007 propose a system for automatically generating text-based signatures for botnet detection. The system works by extracting common strings from observed traffic, then ranking the collected strings by first clustering network traces, and then calculating the information gain for each string based on the entropy of its appearance in clusters. The highly ranked strings can then be used for generating signatures. The system is tested on traffic collected from 1.4 million malware samples. After manually analyzing the top 100 strings, the system found 29 good signatures, 41 signatures for benign traffic, and 30 unknown, which is an improvement on previous work BIB001 . Further, BIB006 proposed a system for large-scale automatic signature generation (ASG). The system uses network traces collected from sandboxes and produces signatures for groups of similar malware, covering numerous protocols. This system is able to identify numerous malware examples with a high TP rate and experiences a low FP rate due to the specificness of the signatures generated. The signatures are designed to be exported to IDSs such as Snort for use in online detection. BotHunter BIB002 ] is a system for identifying compromised hosts based on the actions that they perform, more specifically the pattern of infection and initial connection to a C&C server. There are five steps to this pattern: inbound scan, inbound exploit, binary download, outbound C&C communication, and outbound infection scanning (for propagation). These steps are identified as being a good generalization of the typical infection model for a botnet. Detection is performed by looking for combinations of these actions within a certain time period. Tested on a live campus network, the system is able to achieve a 95% TP rate and low FP rate.