Common test generators fall into two categories. Generating test inputs at the unit level is fast, but can lead to false alarms when a function is called with inputs that would not occur in a system context. If a generated input at the system level causes a failure, this is a true alarm, as the input could also have come from the user or a third party; but system testing is much slower. In this paper, we introduce the concept of a test generation bridge, which joins the accuracy of system testing with the speed of unit testing. A Test Generation Bridge allows to combine an arbitrary system test generator with an arbitrary unit test generator. It does so by carving parameterized unit tests from system (test) executions. These unit tests run in a context recorded from the system test, but individual parameters are left free for the unit test generator to systematically explore. This allows symbolic test generators such as KLEE to operate on individual functions in the recorded system context. If the test generator detects a failure, we lift the failure-inducing parameter back to the system input; if the failure can be reproduced at the system level, it is reported as a true alarm. Our BASILISK prototype can extract and test units out of complex systems such as a Web Python SQLite C stack; in its evaluation, it achieves a higher coverage than a state-of-the-art system test generator.
Compared to the system level, test generation at the unit level is very efficient, as a function call takes less time than a system invocation or interaction; furthermore, exhaustive and symbolic techniques are easier to deploy due to the smaller scale. The downside is that generated function calls may lack realistic context, which makes exploration harder; and function failures may be false alarms because of violated implicit preconditions. @cite_11 devised a technique to use the static calling context of a function in unit-testing. They report high bug detection ability and high precision, but still some false positives. In contrast, we derive the context from execution, that is, with a dynamic analysis. Also, validating all unit-level failures at the system level means that we can recover from false alarms and any remaining failures are true failures.
@cite_11
2885994992
Automated unit testing reduces manual effort to write unit test drivers stubs and generate unit test inputs. However, automatically generated unit test drivers stubs raise false alarms because they often over-approximate real contexts of a target function f and allow infeasible executions of f. To solve this problem, we have developed a concolic unit testing technique CONBRIO. To provide realistic context to f, it constructs an extended unit of f that consists of f and closely relevant functions to f. Also, CONBRIO filters out a false alarm by checking feasibility of a corresponding symbolic execution path with regard to f's symbolic calling contexts obtained by combining symbolic execution paths of f's closely related predecessor functions. In the experiments on the crash bugs of 15 real-world C programs, CONBRIO shows both high bug detection ability (i.e. 91.0 of the target bugs detected) and high precision (i.e. a true to false alarm ratio is 1:4.5). Also, CONBRIO detects 14 new bugs in 9 target C programs studied in papers on crash bug detection techniques.