An important problem of cyber-security is malware analysis. Besides good precision and recognition rate, ideally, a malware detection scheme needs to be able to generalize well for novel malware families (a.k.a zero-day attacks). It is important that the system does not require excessive computation particularly for deployment on the mobile devices. In this paper, we propose a novel scheme to detect malware which we call Malytics. It is not dependent on any particular tool or operating system. It extracts static features of any given binary file to distinguish malware from benign. Malytics consists of three stages: feature extraction, similarity measurement, and classification. The three phases are implemented by a neural network with two hidden layers and an output layer. We show feature extraction, which is performed by tf -simhashing, is equivalent to the first layer of a particular neural network. We evaluate Malytics performance on both Android and Windows platforms. Malytics outperforms a wide range of learning-based techniques and also individual state-of-the-art models on both platforms. We also show Malytics is resilient and robust in addressing zero-day malware samples. The F1-score of Malytics is 97.21 and 99.45 on Android dex file and Windows PE files, respectively, in the applied datasets. The speed and efficiency of Malytics are also evaluated.
ELM shows very promising results for malware activity detection and identification of malicious packed executables. Kozi @cite_36 presented a distributed ELM using NetFlow data structure alongside the Apache Spark https: spark.apache.org framework that provided good performance. Different types of ELM have been applied for malicious packed executable identification @cite_46 .
@cite_36 @cite_46
2767620858 2121749752 2768073432 2243197955
Abstract The Netflow protocol is often used for network auditing, analysis, and monitoring. However, it also can be successfully used as a reliable source of information for incidents detection and forensic purposes. In this paper, the method that combines NetFlows with Extreme Learning Machines (ELM) classifier trained in a distributed environment of Apache Spark framework is proposed. The main contribution of this research is an algorithm that leverages Map-Reduce programming model to scale and distribute a training process of an ELM classifier for a NetFlow-based malware activities detection. Results reported on a benchmark dataset show that the proposed ELM-based NetFlow analysis can be considered as a reliable tool for a network incidents detection. We describe the use of machine learning and data mining to detect and classify malicious executables as they appear in the wild. We gathered 1,971 benign and 1,651 malicious executables and encoded each as a training example using n-grams of byte codes as features. Such processing resulted in more than 255 million distinct n-grams. After selecting the most relevant n-grams for prediction, we evaluated a variety of inductive methods, including naive Bayes, decision trees, support vector machines, and boosting. Ultimately, boosted decision trees outperformed other methods with an area under the ROC curve of 0.996. Results suggest that our methodology will scale to larger collections of executables. We also evaluated how well the methods classified executables based on the function of their payload, such as opening a backdoor and mass-mailing. Areas under the ROC curve for detecting payload function were in the neighborhood of 0.9, which were smaller than those for the detection task. However, we attribute this drop in performance to fewer training examples and to the challenge of obtaining properly labeled examples, rather than to a failing of the methodology or to some inherent difficulty of the classification task. Finally, we applied detectors to 291 malicious executables discovered after we gathered our original collection, and boosted decision trees achieved a true-positive rate of 0.98 for a desired false-positive rate of 0.05. This result is particularly important, for it suggests that our methodology could be used as the basis for an operational system for detecting previously undiscovered malicious executables. The emergence of malicious apps poses a serious threat to the Android platform. Most types of mobile malware rely on network interface to coordinate operations, steal users' private information, and launch attack activities. In this paper, we propose an effective and automatic malware detection method using the text semantics of network traffic. In particular, we consider each HTTP flow generated by mobile apps as a text document, which can be processed by natural language processing to extract text-level features. Then, we use the text semantic features of network traffic to develop an effective malware detection model. In an evaluation using 31 706 benign flows and 5258 malicious flows, our method outperforms the existing approaches, and gets an accuracy of 99.15 . We also conduct experiments to verify that the method is effective in detecting newly discovered malware, and requires only a few samples to achieve a good detection result. When the detection model is applied to the real environment to detect unknown applications in the wild, the experimental results show that our method performs significantly better than other popular anti-virus scanners with a detection rate of 54.81 . Our method also reveals certain malware types that can avoid the detection of anti-virus scanners. In addition, we design a detection system on encrypted traffic for bring-your-own-device enterprise network, home network, and 3G 4G mobile network. The detection model is integrated into the system to discover suspicious network behaviors. This paper presents a system for detecting intrusions when analyzing the network traffic payload looking for malware evidences. The system implements the detection algorithm as a Snort preprocessor component. Since they work together, a highly effective system against known attacks has been achieved (based on Snort rules) and a highly effective system against unknown threats (which was the main aim of the designed system). As the majority of such systems, the proposal consists of two phases: a training phase and a detection phase. During the training phase a statistical model of the legitimate network usage is created through Bloom Filters and N-grams techniques. Subsequently, the results obtained by analyzing a dataset of attacks are compared with such model. This will allow a set of rules to be developed which will be able to determine whether the packets payloads contain malware. In the detection phase, the traffic to analyze is compared with the model created in the training phase and the results obtained when applying rules. The performed experiments showed really satisfactory results, with 100 malware detection and just 0.15 false positives.