Appropriate test data is a crucial factor to reach success in dynamic software testing, e.g., fuzzing. Most of the real-world applications, however, accept complex structure inputs containing data surrounded by meta-data which is processed in several stages comprising of the parsing and rendering (execution). It makes the automatically generating efficient test data, to be non-trivial and laborious activity. The success of deep learning to cope in solving complex tasks especially in generative tasks has motivated us to exploit it in the context of complex test data generation. To do so, a neural language model (NLM) based on deep recurrent neural networks (RNNs) is used to learn the structure of complex input. Our approach generates new test data while distinguishes between data and meta-data that makes it possible to target both the parsing and rendering parts of software under test (SUT). Such test data can improve, input fuzzing. To assess the proposed approach, we developed a modular file format fuzzer, IUST-DeepFuzz. Our conducted experiments on the MuPDF, a lightweight and favorite portable document format (PDF) reader, reveal that IUST-DeepFuzz reaches high coverage of SUT in comparison with the state-of-the-art tools such as learn&fuzz, AFL, Augmented-AFL and random fuzzing. We also observed that the simpler deep learning models, the higher code coverage.
In this section, we discuss some related works in fuzzing and explain their existing problems concerning test data generation. According to the test data generation methods, fuzzers are categorized as Mutation-based and generation-based @cite_29 @cite_28 @cite_34 . Various techniques are applied to both methods to improve them. Most of these techniques have focused on artificial intelligence algorithms.
@cite_28 @cite_29 @cite_34
2808279976 2769748476 2949986519 2128006558
Random program generation — fuzzing — is an effective technique for discovering bugs in compilers but successful fuzzers require extensive development effort for every language supported by the compiler, and often leave parts of the language space untested. We introduce DeepSmith, a novel machine learning approach to accelerating compiler validation through the inference of generative models for compiler inputs. Our approach infers a learned model of the structure of real world code based on a large corpus of open source code. Then, it uses the model to automatically generate tens of thousands of realistic programs. Finally, we apply established differential testing methodologies on them to expose bugs in compilers. We apply our approach to the OpenCL programming language, automatically exposing bugs with little effort on our side. In 1,000 hours of automated testing of commercial and open source compilers, we discover bugs in all of them, submitting 67 bug reports. Our test cases are on average two orders of magnitude smaller than the state-of-the-art, require 3.03× less time to generate and evaluate, and expose bugs which the state-of-the-art cannot. Our random program generator, comprising only 500 lines of code, took 12 hours to train for OpenCL versus the state-of-the-art taking 9 man months to port from a generator for C and 50,000 lines of code. With 18 lines of code we extended our program generator to a second language, uncovering crashes in Solidity compilers in 12 hours of automated testing. Fuzzing is a popular dynamic program analysis technique used to find vulnerabilities in complex software. Fuzzing involves presenting a target program with crafted malicious input designed to cause crashes, buffer overflows, memory errors, and exceptions. Crafting malicious inputs in an efficient manner is a difficult open problem and often the best approach to generating such inputs is through applying uniform random mutations to pre-existing valid inputs (seed files). We present a learning technique that uses neural networks to learn patterns in the input files from past fuzzing explorations to guide future fuzzing explorations. In particular, the neural models learn a function to predict good (and bad) locations in input files to perform fuzzing mutations based on the past mutations and corresponding code coverage information. We implement several neural models including LSTMs and sequence-to-sequence models that can encode variable length input files. We incorporate our models in the state-of-the-art AFL (American Fuzzy Lop) fuzzer and show significant improvements in terms of code coverage, unique code paths, and crashes for various input formats including ELF, PNG, PDF, and XML. Fuzzing consists of repeatedly testing an application with modified, or fuzzed, inputs with the goal of finding security vulnerabilities in input-parsing code. In this paper, we show how to automate the generation of an input grammar suitable for input fuzzing using sample inputs and neural-network-based statistical machine-learning techniques. We present a detailed case study with a complex input format, namely PDF, and a large complex security-critical parser for this format, namely, the PDF parser embedded in Microsoft's new Edge browser. We discuss (and measure) the tension between conflicting learning and fuzzing goals: learning wants to capture the structure of well-formed inputs, while fuzzing wants to break that structure in order to cover unexpected code paths and find bugs. We also present a new algorithm for this learn&fuzz challenge which uses a learnt input probability distribution to intelligently guide where to fuzz inputs. We present an extension of traditional "black box" fuzz testing using a genetic algorithm based upon a dynamic Markov model fitness heuristic. This heuristic allows us to "intelligently" guide input selection based upon feedback concerning the "success" of past inputs that have been tried. Unlike many software testing tools, our implementation is strictly based upon binary code and does not require that source code be available. Our evaluation on a Windows server program shows that this approach is superior to random black box fuzzing for increasing code coverage and depth of penetration into program control flow logic. As a result, the technique may be beneficial to the development of future automated vulnerability analysis tools.