For a few program versions NUMFL failed to assign a suspiciousness score to any assignment statement and the suspiciousness list only contained zero values. We suspect this is caused by using a binary outcome variable rather than the absolute difference between the actual and expected output as NUMFL is intended to do. We did not include these versions in our comparison. We also did not include a program version if it had fewer than 20 “relevant” test cases, which cover a faulty class [1]}. The remaining subject program versions are summarized in Table REF .
<TABLE>