For years security machine learning research has promised to obviate the need for signature based detection by automatically learning to detect indicators of attack. Unfortunately, this vision hasn't come to fruition: in fact, developing and maintaining today's security machine learning systems can require engineering resources that are comparable to that of signature-based detection systems, due in part to the need to develop and continuously tune the "features" these machine learning systems look at as attacks evolve. Deep learning, a subfield of machine learning, promises to change this by operating on raw input signals and automating the process of feature design and extraction. In this paper we propose the eXpose neural network, which uses a deep learning approach we have developed to take generic, raw short character strings as input (a common case for security inputs, which include artifacts like potentially malicious URLs, file paths, named pipes, named mutexes, and registry keys), and learns to simultaneously extract features and classify using character-level embeddings and convolutional neural network. In addition to completely automating the feature design and extraction process, eXpose outperforms manual feature extraction based baselines on all of the intrusion detection problems we tested it on, yielding a 5 -10 detection rate gain at 0.1 false positive rate compared to these baselines.
With respect to the detection mechanism used in previous URL detection work, the simplest proposed approaches have involved blacklists, which can be collected using manual labeling, user feedback, client honeypots, and other heuristics @cite_21 . While blacklists have a very low false positive rate, they are also very brittle and thus cannot generalize to previously unseen URL strings @cite_27 . To address these limitations, statistical approaches, such as machine-learning or similarity based URL detection have been proposed @cite_15 . Unfortunately, manually discovering potentially useful features is time consuming and requires constant adaptation to evolving obfuscation techniques, which limits the achievable accuracy of the detectors. In contrast to work that requires manual feature extraction from URLs to make detections, our work automates this feature extraction process.
@cite_27 @cite_21 @cite_15
2160353917 2057871674 2028223155
In this paper, we study the eectiveness of phishing blacklists. We used 191 fresh phish that were less than 30 minutes old to conduct two tests on eight anti-phishing toolbars. We found that 63 of the phishing campaigns in our dataset lasted less than two hours. Blacklists were ineective when protecting users initially, as most of them caught less than 20 of phish at hour zero. We also found that blacklists were updated at dierent speeds, and varied in coverage, as 47 - 83 of phish appeared on blacklists 12 hours from the initial test. We found that two tools using heuristics to complement blacklists caught signicantly more phish initially than those using only blacklists. However, it took a long time for phish detected by heuristics to appear on blacklists. Finally, we tested the toolbars on a set of 13,458 legitimate URLs for false positives, and did not nd any instance of mislabeling for either blacklists or heuristics. We present these ndings and discuss ways in which anti-phishing tools can be improved. Detecting malicious URLs is an essential task in network security intelligence. In this paper, we make two new contributions beyond the state-of-the-art methods on malicious URL detection. First, instead of using any pre-defined features or fixed delimiters for feature selection, we propose to dynamically extract lexical patterns from URLs. Our novel model of URL patterns provides new flexibility and capability on capturing malicious URLs algorithmically generated by malicious programs. Second, we develop a new method to mine our novel URL patterns, which are not assembled using any pre-defined items and thus cannot be mined using any existing frequent pattern mining methods. Our extensive empirical study using the real data sets from Fortinet, a leader in the network security industry, clearly shows the effectiveness and efficiency of our approach. This article surveys the literature on the detection of phishing attacks. Phishing attacks target vulnerabilities that exist in systems due to the human factor. Many cyber attacks are spread via mechanisms that exploit weaknesses found in end-users, which makes users the weakest element in the security chain. The phishing problem is broad and no single silver-bullet solution exists to mitigate all the vulnerabilities effectively, thus multiple techniques are often implemented to mitigate specific attacks. This paper aims at surveying many of the recently proposed phishing mitigation techniques. A high-level overview of various categories of phishing mitigation techniques is also presented, such as: detection, offensive defense, correction, and prevention, which we belief is critical to present where the phishing detection techniques fit in the overall mitigation process.