On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Other Attacks. <s> Owed to their versatile functionality and widespread adoption, PDF documents have become a popular avenue for user exploitation ranging from large-scale phishing attacks to targeted attacks. In this paper, we present a framework for robust detection of malicious documents through machine learning. Our approach is based on features extracted from document metadata and structure. Using real-world datasets, we demonstrate the the adequacy of these document properties for malware detection and the durability of these features across new malware variants. Our analysis shows that the Random Forests classification method, an ensemble classifier that randomly selects features for each individual classification tree, yields the best detection rates, even on previously unseen malware. Indeed, using multiple datasets containing an aggregate of over 5,000 unique malicious documents and over 100,000 benign ones, our classification rates remain well above 99% while maintaining low false positives of 0.2% or less for different classification parameters and experimental scenarios. Moreover, the classifier has the ability to detect documents crafted for targeted attacks and separate them from broadly distributed malicious PDF documents. Remarkably, we also discovered that by artificially reducing the influence of the top features in the classifier, we can still achieve a high rate of detection in an adversarial setting where the attacker is aware of both the top features utilized in the classifier and our normality model. Thus, the classifier is resilient against mimicry attacks even with knowledge of the document features, classification method, and training set. <s> BIB001 </s> On the Security of Machine Learning in Malware C&C Detection: A Survey <s> Other Attacks. <s> Classifier evasion consists in finding for a given instance $x$ the nearest instance $x'$ such that the classifier predictions of $x$ and $x'$ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixed Integer Linear Program solver and finds the optimal evading instance under an expressive set of constraints. Our second algorithm trades off optimality for speed by using symbolic prediction, a novel algorithm for fast finite differences on tree ensembles. On a digit recognition task, we demonstrate that both gradient boosted trees and random forests are extremely susceptible to evasions. Finally, we harden a boosted tree model without loss of predictive accuracy by augmenting the training set of each boosting round with evading instances, a technique we call adversarial boosting. <s> BIB002
Genetic programming.  apply a genetic programming approach to evading PDF malware classifiers, more specifically PDFRate BIB001 and Hidost [Šrndic and Laskov 2013] systems. PDFRate makes use of the random forest classifier (as described in Section 6.1.1), whereas Hidost makes use of SVM. Hidost, according to its authors, is robust against adversaries, only suffering from two additional FPs (out of 5,000 samples) under the "strongest conceivable mimicry attack". In contrast to previous work, the attack assumes no knowledge on behalf of the attacker, with the attacker only having access to a black box implementation of the target (the deployed system). Also in contrast to previous work, where it is assumed that attackers can only increase feature values (to maintain malicious behavior), the attacker in this scenario is able to modify feature values in many ways. The attack works by generating attack traces that represent series of operations of the original file to cause misclassification. Traces are computed using an iterative approach, with random operations applied to the file (e.g., inserting a page from a benign file). The modifications are evaluated using a fitness function, which incorporates the score from the target detector, and whether or not the file still carries out its intended malicious behavior (evaluated using an oracle in the form of a sandbox). The attack is instantiated by taking a set of malicious files as an initial population and then applying the modifications. Previously found successful traces are applied to some files to create new starting points to aid in finding attacks. The attack is tested on 1,348 malicious PDF files and is shown to be able to evade both PDFRate and Hidost with a 100% success rate. Attack traces generated for Hidost, but submitted to PDFRate, were able to evade detection in 77.6% of cases, whereas in the reverse case, only two seeds were able to evade detection. The authors speculate that this is due to the different feature sets in use by each system. Tree ensemble evasion. BIB002 propose a method of evading tree ensemble-based classifiers such as boosted trees and random forest. Two approaches are proposed: an optimal case using a mixed integer linear program (MILP) solver and a lighter approximate evasion algorithm based on iterative coordinate descent for use on more complex ensemble models where the optimal approach is too time consuming. The goal is to find the attack point that changes the classifier output, where the distance from the original is minimized according to one of four distance The attack is tested using the MNIST handwritten digits dataset, tested against boosted decision trees and random forest, plus a linear LR, a simple neural network, SVM with the RBF kernel, and a shallow neural network based on convex polytope machines. Although the boosted trees and random forest are shown to have competitive accuracies when not under attack, they are shown to be far less robust than the other classifiers when under attack, requiring a smaller evasion distance, particularly in the L 1 and L 2 cases, where the evasion distance is negligible. The authors also propose a method for improving robustness by including attack samples in the training data; however, it is found that when providing strong resilience to L − 0 attacks, the classifier becomes less robust to the other types.