What is the impact of software engineering research on current practices in industry? In this paper, I report on my direct experience as a PhD post-doc working in software engineering research projects, and then spending the following five years as an engineer in two different companies (the first one being the same I worked in collaboration with during my post-doc). Given a background in software engineering research, what cutting-edge techniques and tools from academia did I use in my daily work when developing and testing the systems of these companies? Regarding validation and verification (my main area of research), the answer is rather short: as far as I can tell, only FindBugs. In this paper, I report on why this was the case, and discuss all the challenging, complex open problems we face in industry and which somehow are “neglected” in the academic circles. In particular, I will first discuss what actual tools I could use in my daily work, such as JaCoCo and Selenium. Then, I will discuss the main open problems I faced, particularly related to environment simulators, unit and web testing. After that, popular topics in academia are presented, such as UML, regression and mutation testing. Their lack of impact on the type of projects I worked on in industry is then discussed. Finally, from this industrial experience, I provide my opinions about how this situation can be improved, in particular related to how academics are evaluated, and advocate for a greater involvement into open-source projects.
The level of industry-academia collaboration (IAC) in software engineering in general and in software testing in particular is quite low. Many researchers and practitioners are not collaborating with the "other side" to solve industrial problems. To shed light on the above issue and to characterize precisely what industry wants from academia in software testing, we solicited practitioners' opinions on their challenges in different testing activities and also the particularly relevant topics that they want the research community to work on. This short paper aims to draw the community's attention to the important issue of strengthening IAC with the hope of more IAC in software testing in the areas of most importance to the industry. Collaboration between industry and academia supports improvement and innovation in industry and helps to ensure industrial relevance in academic research. However, many researchers and practitioners believe that the level of joint industry–academia collaborations (IAC) in software engineering (SE) is still relatively very low, compared to the amount of activity in each of the two communities. The goal of the empirical study reported in this paper is to characterize a set of collaborative industry–academia R&D projects in the area of software testing conducted by the authors (based in Canada and Turkey) with respect to a set of challenges, patterns and anti-patterns identified by a recent Systematic Literature Review study, with the aim of contributing to the body of evidence in the area of IAC, for the benefit of SE researchers and practitioners in conducting successful IAC projects in software testing and in software engineering in general. To address the above goal, a pool of ten IAC projects (six completed, two failed and two ongoing) all in the area of software testing, which the authors have led or have had active roles in, were selected as objects of study and were analyzed (both quantitatively and qualitatively) with respect to the set of selected challenges, patterns and anti-patterns. As outputs, the study presents a set of empirical findings and evidence-based recommendations, e.g.: it has been observed that even if an IAC project may seem perfect from many aspects, one single major challenge (e.g., disagreement in confidentiality agreements) can lead to its failure. Thus, we recommend that both parties (academics and practitioners) consider all the challenges early on and proactively work together to eliminate the risk of challenges in IAC projects. We furthermore report correlation and interrelationship of challenges, patterns and anti-patterns with project success measures. This study hopes to encourage and benefit other SE researchers and practitioners in conducting successful IAC projects in software testing and in software engineering in general in the future. Context: The global software industry and the software engineering (SE) academia are two large communities. However, unfortunately, the level of joint industry-academia collaborations in SE is still relatively very low, compared to the amount of activity in each of the two communities. It seems that the two 'camps' show only limited interest motivation to collaborate with one other. Many researchers and practitioners have written about the challenges, success patterns (what to do, i.e., how to collaborate) and anti-patterns (what not do do) for industry-academia collaborations.Objective: To identify (a) the challenges to avoid risks to the collaboration by being aware of the challenges, (b) the best practices to provide an inventory of practices (patterns) allowing for an informed choice of practices to use when planning and conducting collaborative projects.Method: A systematic review has been conducted. Synthesis has been done using grounded-theory based coding procedures.Results: Through thematic analysis we identified 10 challenge themes and 17 best practice themes. A key outcome was the inventory of best practices, the most common ones recommended in different contexts were to hold regular workshops and seminars with industry, assure continuous learning from industry and academic sides, ensure management engagement, the need for a champion, basing research on real-world problems, showing explicit benefits to the industry partner, be agile during the collaboration, and the co-location of the researcher on the industry side.Conclusion: Given the importance of industry-academia collaboration to conduct research of high practical relevance we provide a synthesis of challenges and best practices, which can be used by researchers and practitioners to make informed decisions on how to structure their collaborations.