Most requirements engineering (RE) process improvement approaches are solution-driven and activity-based. They focus on the assessment of the RE of a company against an external norm of best practices. A consequence is that practitioners often have to rely on an improvement approach that skips a profound problem analysis and that results in an RE approach that might be alien to the organisational needs. In recent years, we have developed an RE improvement approach (called ) that guides a holistic RE improvement against individual goals of a company putting primary attention to the quality of the artefacts. In this paper, we aim at exploring ArtREPI's benefits and limitations. We contribute an industrial evaluation of ArtREPI by relying on a case study research. Our results suggest that ArtREPI is well-suited for the establishment of an RE that reflects a specific organisational culture but to some extent at the cost of efficiency resulting from intensive discussions on a terminology that suits all involved stakeholders. Our results reveal first benefits and limitations, but we can also conclude the need of longitudinal and independent investigations for which we herewith lay the foundation.
In response to their shortcoming, contributed an approach to problem-driven RE improvement @cite_4 called the iFLAP approach. Same as in ArtREPI, they make use of qualitative methods for the problem analysis and postulate the importance of strong stakeholder involvement. Although their concepts are promising to conduct a problem-driven REPI, the consequential next steps, i.e. the actual improvement realisation by crafting a new RE reference model, was not in scope of their contribution.
@cite_4
177853461 2963686848 1496688783 1539975474
Requirements engineering process improvement (REPI) has gained much attention in research and practice. Most REPI approaches are of solution-driven and activity-based nature. They focus on the assessment of company-specific RE reference models against an external norm of best practices, and they propagate an improvement by forecasting the adaptation of the processes and methods in the RE reference model towards that norm. In recent years, we could develop a first problem-driven RE improvement approach that supports an improvement against individual goals and problems of a company putting primary attention to the quality of the RE artefacts (named ArtREPI). In this short paper, we briefly illustrate our resulting approach and report on our initial experiences from ongoing empirical evaluations in practice. We conclude with a summary of planned next steps. Abstract Context For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, they still do not allow for empirical generalisations. Objective To lay an empirical and externally valid foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method We designed a globally distributed family of surveys in joint collaborations with different researchers and completed the first run in Germany. The instrument is based on a theory in the form of a set of hypotheses inferred from our experiences and available studies. We test each hypothesis in our theory and identify further candidates to extend the theory by correlation and Grounded Theory analysis. Results In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from Germany with participants from 58 companies. The results reveal, for example, a tendency to improve RE via internally defined qualitative methods rather than relying on normative approaches like CMMI. We also discovered various RE problems that are statistically significant in practice. For instance, we could corroborate communication flaws or moving targets as problems in practice. Our results are not yet fully representative but already give first insights into current practices and problems in RE, and they allow us to draw lessons learnt for future replications. Conclusion Our results obtained from this first run in Germany make us confident that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice. The importance of continuously improving requirements engineering (RE) has been recognised for many years. Similar to available software process improvement approaches, most RE improvement approaches focus on a normative and solution-driven assessment of companies rather than on a problem-driven RE improvement. The approaches dictate the implementation of a one-size-fits-all reference model without doing a proper problem investigation first, whereas the notion of quality factually depends on whether RE achieves company-specific goals. The approaches furthermore propagate process areas and methods, without proper awareness of the quality in the created artefacts on which the quality of many development phases rely. Little knowledge exists about how to conduct a problem-driven RE improvement that gives attention to the improvement of the artefacts. A promising solution is to start an improvement with an empirical investigation of the RE stakeholders, goals, and artefacts in the company to identify problems while abstracting from inherently complex processes. The RE improvement is then defined and implemented in joint action research workshops with the stakeholders to validate potential solutions while again concentrating on the artefacts. In this paper, we contribute an artefact-based, problem-driven RE improvement approach that emerged from a series of completed RE improvements. We discuss lessons learnt and present first result from an ongoing empirical evaluation at a German company. Our results suggest that our approach supports process engineers in a problem-driven RE improvement, but we need deeper examination of the resulting RE company standard, which is in scope of the final evaluation. We consider the problem of incorporating end-user advice into reinforcement learning (RL). In our setting, the learner alternates between practicing, where learning is based on actual world experience, and end-user critique sessions where advice is gathered. During each critique session the end-user is allowed to analyze a trajectory of the current policy and then label an arbitrary subset of the available actions as good or bad. Our main contribution is an approach for integrating all of the information gathered during practice and critiques in order to effectively optimize a parametric policy. The approach optimizes a loss function that linearly combines losses measured against the world experience and the critique data. We evaluate our approach using a prototype system for teaching tactical battle behavior in a real-time strategy game engine. Results are given for a significant evaluation involving ten end-users showing the promise of this approach and also highlighting challenges involved in inserting end-users into the RL loop.