Abstract Bad requirements quality can cause expensive consequences during the software development lifecycle, especially if iterations are long and feedback comes late. We aim at a light-weight static requirements analysis approach that allows for rapid checks immediately when requirements are written down. We transfer the concept of code smells to requirements engineering as Requirements Smells . To evaluate the benefits and limitations, we define Requirements Smells, realize our concepts for a smell detection in a prototype called Smella and apply Smella in a series of cases provided by three industrial and a university context. The automatic detection yields an average precision of 59 at an average recall of 82 with high variation. The evaluation in practical environments indicates benefits such as an increase of the awareness of quality defects. Yet, some smells were not clearly distinguishable. Lightweight smell detection can uncover many practically relevant requirements defects in a reasonably precise way. Although some smells need to be defined more clearly, smell detection provides a helpful means to support quality assurance in requirements engineering, for instance, as a supplement to reviews.
In summary, in our contribution, we extend the current state of reported evidence on automatic QA for requirements artifacts via systematic studies in terms of distribution, precision, recall, and relevance, as well as by means of a systematic evaluation with practitioners under realistic conditions. We perform this on both existing, as well as new quality defects taken from the ISO 29148. Therefore, we extend our previously published first empirical steps @cite_41 to close these gaps by thorough empirical evaluation.
@cite_41
2114647523
There is a documented gap between academic and practitioner views on software testing. This paper tries to close the gap by investigating both views regarding the benefits and limits of test automation. The academic views are studied with a systematic literature review while the practitioners views are assessed with a survey, where we received responses from 115 software professionals. The results of the systematic literature review show that the source of evidence regarding benefits and limitations is quite shallow as only 25 papers provide the evidence. Furthermore, it was found that benefits often originated from stronger sources of evidence (experiments and case studies), while limitations often originated from experience reports. We believe that this is caused by publication bias of positive results. The survey showed that benefits of test automation were related to test reusability, repeatability, test coverage and effort saved in test executions. The limitations were high initial invests in automation setup, tool selection and training. Additionally, 45 of the respondents agreed that available tools in the market offer a poor fit for their needs. Finally, it was found that 80 of the practitioners disagreed with the vision that automated testing would fully replace manual testing.