A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> Merging the art and science of software development <s> BIB001 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> DevOps is an emerging paradigm to actively foster the collaboration between system developers and operations in order to enable efficient end-to-end automation of software deployment and management processes. DevOps is typically combined with Cloud computing, which enables rapid, on-demand provisioning of underlying resources such as virtual servers, storage, or database instances using APIs in a self-service manner. Today, an ever-growing amount of DevOps tools, reusable artifacts such as scripts, and Cloud services are available to implement DevOps automation. Thus, informed decision making on the appropriate approach (es) for the needs of an application is hard. In this work we present a collaborative and holistic approach to capture DevOps knowledge in a knowledgebase. Beside the ability to capture expert knowledge and utilize crowd sourcing approaches, we implemented a crawling framework to automatically discover and capture DevOps knowledge. Moreover, we show how this knowledge is utilized to deploy and operate Cloud applications. <s> BIB002 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> A new discipline at the intersection of the development and operation of software systems known as DevOps has seen significant growth recently. Among the wide range of tasks of DevOps professionals, we focus on that of selecting appropriate cloud deployments for distributed applications. Despite the advent of automated software deployment and management frameworks, reasoning about good deployments still requires interaction with experts, often through discussions on online technical forums and social networks. <s> BIB003 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> DevOps (a portmanteau of “development” and “operations”) is a software development method that extends the agile philosophy to rapidly produce software products and services and to improve operations performance and quality assurance. It was born to accelerate the delivery of web-based systems and quickly bring new value to users. Many web-based systems evolve according to usage trends without a clear long-term goal. Before the widespread use of web services, most software with a clear goal were delivered as packages that users installed on their own system. New versions were delivered with a much lower frequency, with periods in between versions ranging from months to years. Development cycles were divided into large design, coding and testing phases culminating in the release of a new stable version. In software development in the context of applied science, even when the goal is clear, the process to attain it is not. Hence, working releases that capture the current software state must be released frequently in order to reduce the risks for all stakeholders and to make it possible to assess the current state of a project and steer it in the right direction. This paper explores the usefulness of DevOps concepts to improve the development of software that supports scientific projects. We establish the similarities and differences between scientific projects and web applications development, and discuss where the related methodologies need to be extended. Unique challenges are discussed herewith developed solutions, and still open questions. Lessons learned are highlighted as best practices to be followed in research projects. This discussion is rooted in our experience in real-life projects at the IBM Research Brazil Lab, which just as well apply to other research institutions. <s> BIB004 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> Context: Development and Operations (DevOps) is an emerging software industry movement to bridge the gap between software development and operations teams. DevOps supports frequently and reliably releasing new features and products-- thus subsuming Continuous Deployment (CD) practice. Goal: This research aims at empirically exploring the potential impact of CD practice on architecting process. Method: We carried out a case study involving interviews with 16 software practitioners. Results: We have identified (1) a range of recurring architectural challenges (i.e., highly coupled monolithic architecture, team dependencies, and ever-changing operational environments and tools) and (2) five main architectural principles (i.e., small and independent deployment units, not too much focus on reusability, aggregating logs, isolating changes, and testability inside the architecture) that should be considered when an application is (re-) architected for CD practice. This study also supports that software architecture can better support operations if an operations team is engaged at an early stage of software development for taking operational aspects into considerations. Conclusion: These findings provide evidence that software architecture plays a significant role in successfully and efficiently adopting continuous deployment. The findings contribute to establish an evidential body of knowledge about the state of the art of architecting for CD practice <s> BIB005 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> "Software security" often evokes negative feelings amongst software developers because this term is associated with additional programming effort, uncertainty and road blocker activity on rapid development and release cycles. The Secure DevOps movement attempts to combat the toxic environment surrounding software security by shifting the paradigm from following rules and guidelines to creatively determining solutions for tough security problems Taschner, 2015. Secure software should be focused on a proactive approach that limits the attack surface and produces reliable software. Secure DevOps developers want their software to bend but not break, which means the software absorbs attacks and continues to function. The burgeoning concepts of DevOps include a number of concepts that can be applied to increase the security of developed applications. Applying these and other DevOps principles can have a big impact on creating an environment that is resilient and secure. Specifically, this paper clearly explains how to address security concerns in the early stages of the development lifecycle and leverage that knowledge throughout the SDLC. <s> BIB006 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> DevOps is a predominant phenomenon in the web domain. Its two core principles emphasize collaboration between software development and operations, and the use of agile principles to manage deployment environments and their configurations. DevOps techniques, such as collaboration and behaviour-driven monitoring, have been used by web companies to facilitate continuous deployment of new functionality to customers. The techniques may also offer opportunities for continuous product improvement when adopted in the embedded systems domain. However, certain characteristics of embedded software development present obstacles for DevOps adoption, and as yet, there is no empirical evidence of its adoption in the embedded systems domain. In this study, we present the challenges for DevOps adoption in embedded systems using a multiple-case study approach with four companies. The contribution of this paper is to introduce the concept of DevOps adoption in the embedded systems domain and then to identify key challenges for the DevOps adoption. <s> BIB007 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> Even though agile actively seeks collaboration from all its stakeholders, most agile projects do not extend themselves toward the operations people. To solve this problem, DevOps is introduced. DevOps is a conceptual framework for reintegrating development and operations of Information Systems, which is able to break the wall between developers and operations professionals. DevOps improves the work through a collection of principles and practices, centered around close collaboration between Development and Operations personnel. However, both sides have paid little attention to issues faced by each other. Communication gaps is a recurrent problem in agile teams that is also eminent in the relationship between developers and operations. Literature offers little research on this aspect of communication in DevOps. This position paper describes the communication practices from a distributed agile team composed of developers and operations based on communication challenges (geographical, socio-cultural, and temporal distance) and strategies (frequency, direction, modality, and content). From the results we outline possible research focus for future work, aiming to enrich the academia research on the matter as well as to help practitioners to improve their working practices. <s> BIB008 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> DevOps is a new way of developing software that is challenging from a teaching perspective. In this paper, we outline these challenges and propose teaching methods that focus on skill acquisition and technical practices that focus on performant virtualization to overcome them. We describe central elements from our course Cloud Computing and Architecture that has been designed and executed upon these methods and practices and report our experiences and lessons learned. <s> BIB009 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> The purpose of this research is to propose what fundamental Knowledge (K), Skills (S), and Abilities (A) should be taught in web development courses, which can support not only development but also operations, namely DevOps. Instead of focusing on client side web application development using standard web technologies, we focus on server side web application development which holds many challenges due to a diversity of languages and emerging technologies. The advent of cloud computing reduced the gap between web application development and deployment/operation. Also, it brought an emerging concept, DevOps, to the web application development community. We identify what KSAs are needed to support DevOps and attempt to include them in web development courses. When the KSAs are included in web application development courses, students are well prepared for the emerging practice and computing paradigm of DevOps. <s> BIB010 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> Many software engineering organizations around the world are adopting DevOps. One of the goals of DevOps is to foster better collaboration between development and operations personnel, in order to improve organizational efficiency. Since DevOps is lacking a common definition, there are several approaches to adopt it, and organizations largely need to determine how to apply DevOps for themselves. In this paper, we present results from a case study in which a software organization adopts DevOps. The focus of this research is to study the impact of mixing the responsibilities between development and operations engineers. We interviewed 14 employees in the organization during the study, and results indicate several benefits of the chosen approach, such as improved collaboration and trust, and smoother work flow. This comes at the cost of a number of complications, such as new sources for friction among the employees, risk for holistically sub-optimal service configurations, and more. <s> BIB011 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> Your biggest mistake might be collecting the wrong data. <s> BIB012 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> DevOps and continuous development are getting popular in the software industry. Adopting these modern approaches in regulatory environments, such as medical device software, is not straightforward because of the demand for regulatory compliance. While DevOps relies on continuous deployment and integration, regulated environments require strict audits and approvals before releases. Therefore, the use of modern development approaches in regulatory environments is rare, as is the research on the topic. However, as software is more and more predominant in medical devices, modern software development approaches become attractive. This paper discusses the fit of DevOps for regulated medical device software development. We examine two related standards, IEC 62304 and IEC 82304-1, for obstacles and benefits of using DevOps for medical device software development. We found these standards to set obstacles for continuous delivery and integration. Respectively, development tools can help fulfilling the requirements of traceability and documentation of these standards. <s> BIB013 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> The DevOps phenomenon is gaining popularity through its ability to support continuous value delivery and ready accommodation of change. However, given the relative immaturity and general confusion about DevOps, a common view of expectations from a DevOps role is lacking. Through investigation of online job advertisements, combined with interviews, we identified key Knowledge Areas, Skills and Capabilities for a DevOps role and their relative importance in New Zealand's job market. Our analysis also revealed the global dimensions and the emerging nature of the DevOps role in GSE projects. This research adds a small advanced economy (New Zealand) perspective to the literature on DevOps job advertisements and should be of value to employers, job seekers, researchers as well educators and policy makers. <s> BIB014 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> Over the past three years, Fannie Mae IT has transformed from a traditional waterfall organization to a lean culture enabled by agile methods and DevOps. Software analytics were used to guide improvements and evaluate progress. Project-level analytics enabled agile teams to improve structural quality and evaluate their practices as they delivered greater functionality over shrinking delivery intervals. Aggregated enterprise metrics displayed an average 38 percent improvement in structural quality, with some applications achieving 48 percent to 70 percent gains. These quality improvements accompanied a 28 percent increase in productivity. The frequency of releases shrank from cycles of nine to 18 months, down to releases every couple of one- to two-month sprints. The article discusses examples of how analytics were used, along with challenges in selecting measures and implementing analytics in an agile–DevOps transformation. This article is part of a special issue on Actionable Analytics for Software Engineering. <s> BIB015 </s> A Survey of DevOps Concepts and Challenges <s> Peer-reviewed literature <s> For many software development teams, the first aspects that come to mind regarding continuous delivery (CD) are the operational challenges and competitive benefits. In the authors’ experience, CD was much more: it was a survival technique. This article presents how and why they applied CD in a large governmental project for the development of a collaborative development environment. They share the challenges they faced and the strategies they used to overcome them. The article concludes with a set of lessons learned that can be valuable for readers in a variety of situations. This article is part of a special issue on release engineering. <s> BIB016
This survey primarily relies on evaluating peer-reviewed papers, which is standard practice in the academic community for guaranteeing the quality and credibility of published works. Figure 2 depicts the evolution of published works on DevOps according to the criteria defined in our study design (papers collected in September 2018). Similar to most research areas, conferences generally publish preliminary results. In contrast, journals publish more mature studies. Academic journals primarily reach an academic audience, while magazines also have practitioners as their readers. The "IEEE Software" magazine was a constant source of papers on DevOps, having published 13 of the 50 core papers. They are mostly relatively short papers describing real experiences faced by organizations. Therefore, it is advisable for practitioners to subscribe to this source for the latest updates on DevOps. We did not find any peer-reviewed journal, magazine, or event dedicated exclusively to the theme. The closest related events we found were the "International Workshop on Release Engineering" and the "International Workshop on Quality-Aware DevOps. " For researchers beginning a study on DevOps, these are two potential venues for publishing their first results. The absence of a specific venue indicates an opportunity for the academic community to organize scientific events and journal special issues dedicated to this area. After paper selection, following the protocol presented in Section 3, we classified all 50 papers into seven categories, as shown in Table 1 . The categories are described in the sequence. Practices: provides tools, practices, patterns, and strategies to enhance DevOps. These studies offer guidelines for selecting deployment technology BIB002 , explanation of the use of metrics BIB012 , how to support communication among global peers through specialized social networks BIB003 , and how injecting infrastructure faults at production can support system reliability [3] . Experience Report: refers to reports of organizations adopting DevOps or continuous delivery/deployment. For example, Siqueira et al. describe the impact of a DevOps team in the context of a governmental project BIB016 . Impact: explores how DevOps effects other aspects of software development such as architecture BIB005 , security BIB006 , quality assurance BIB001 , or the impact of DevOps on software development in specific scenarios, such as the development of software for research BIB004 . Concepts: introduces basic concepts in DevOps and continuous delivery, such as the deployment pipeline  . Challenges: covers the challenges entailed in adopting DevOps or continuous delivery/deployment. Some papers focus on specific challenges, such as DevOps in regulated domains BIB013 , embedded systems BIB007 , or communication BIB008 . Education: investigates the challenges in teaching DevOps. Articles in this category propose teaching methods BIB009 , as well as explore the Knowledge, Skills, and Abilities (KSA) necessary for DevOps professionals BIB010 BIB014 . Adoption: covers the different models for DevOps adoption BIB011 and describes how an organization chooses a set of metrics to evaluate its DevOps adoption process BIB015 . This subject is further discussed in Section 8.2.