Randomly generated programs are popular for testing compilers and program analysis tools, with hundreds of bugs in real-world C compilers found by random testing. However, existing random program generators may generate large amounts of dead code (computations whose result is never used). This leaves relatively little code to exercise a target compiler's more complex optimizations. To address this shortcoming, we introduce liveness-driven random program generation. In this approach the random program is constructed bottom-up, guided by a simultaneous structural data-flow analysis to ensure that the generator never generates dead code. The algorithm is implemented as a plugin for the Frama-C framework. We evaluate it in comparison to Csmith, the standard random C program generator. Our tool generates programs that compile to more machine code with a more complex instruction mix.
The JTT program generator @cite_1 is aimed directly at testing compiler optimizations. It uses a model-based approach, where generation is guided using test scripts. These scripts contain code templates and temporal logic specifications of the optimizations to be tested. For example, the authors specify opportunities for dead code elimination as cases where a variable is assigned, then assigned again before being used. The test script contains a temporal logic formula expressing this pattern and the test condition that the compiler should eliminate the first assignment. Using this script, JTT generates test programs containing this pattern. JTT was used successfully to find bugs and increase the test suite's statement coverage for an industrial C compiler.
@cite_1
2141717815
This paper presents joint research and practice on automated test program generation for an industrial compiler, UniPhier, by Matsushita Electric Industrial Co., Ltd. (MEI) and Institute of Software, Chinese Academy of Sciences (ISCAS) since Sept. 2002. To meet the test requirements of MEI's engineers, we proposed an automated approach to produce test programs for UniPhier, and as a result we developed an integrated tool named JTT. Firstly, we show the script-driven test program generation process in JTT. Secondly, we show how to produce test programs automatically, based on a temporal-logic model of compiler optimizations, to guarantee the execution of optimizing modules under test during compilation. JTT has gained success in testing UniPhier: even after benchmark testing and comprehensive manual testing, JTT still found 6 new serious defects.