A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Agile software development is well-known for its focus on close customer collaboration and customer feedback. In emphasizing flexibility, efficiency and speed, agile practices have lead to a paradigm shift in how software is developed. However, while agile practices have succeeded in involving the customer in the development cycle, there is an urgent need to learn from customer usage of software also after delivering and deployment of the software product. The concept of continuous deployment, i.e. the ability to deliver software functionality frequently to customers and subsequently, the ability to continuously learn from real-time customer usage of software, has become attractive to companies realizing the potential in having even shorter feedback loops. However, the transition towards continuous deployment involves a number of barriers. This paper presents a multiple-case study in which we explore barriers associated with the transition towards continuous deployment. Based on interviews at four different software development companies we present key barriers in this transition as well as actions that need to be taken to address these. <s> BIB001 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Rally Software transitioned from shipping code every eight-weeks, with time-boxed Scrum sprints, to a model of continuous delivery with Kanban. The team encountered complex challenges with their build systems, automated test suites, customer enablement, and internal communication. But there was light at the end of the tunnel - greater control and flexibility over feature releases, incremental delivery of value, lower risks, fewer defects, easier on-boarding of new developers, less off-hours work, and a considerable up tick in confidence. This experience report describes the journey to continuous delivery with the aim that others can learn from our mistakes and get their teams deploying more frequently. We will describe and contrast this transition from the business (product management) and engineering perspectives. <s> BIB002 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Abstract Context Continuous Deployment (CD) is an emerging software development process with organisations such as Facebook, Microsoft, and IBM successfully implementing and using the process. The CD process aims to immediately deploy software to customers as soon as new code is developed, and can result in a number of benefits for organisations, such as: new business opportunities, reduced risk for each release, and prevent development of wasted software. There is little academic literature on the challenges organisations face when adopting the CD process, however there are many anecdotal challenges that organisations have voiced on their online blogs. Objective The aim of this research is to examine the challenges faced by organisations when adopting CD as well as the strategies to mitigate these challenges. Method An explorative case study technique that involves in-depth interviews with software practitioners in an organisation that has adopted CD was conducted to identify these challenges. Results This study found a total of 20 technical and social adoption challenges that organisations may face when adopting the CD process. The results are discussed to gain a deeper understanding of the strategies employed by organisations to mitigate the impacts of these challenges. Conclusion While a number of individual technical and social adoption challenges were uncovered by the case study in this research, most challenges were not faced in isolation. The severity of these challenges were reduced by a number of mitigation strategies employed by the case study organisation. It is concluded that organisations need to be well prepared to handle technical and social adoption challenges with their existing expertise, processes and tools before adopting the CD process. For practitioners, knowing how to address the challenges an organisation may face when adopting the CD process provides a level of awareness that they previously may not have had. <s> BIB003 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> As part of a Finnish research program, researchers interviewed 15 information and communications technology companies to determine the extent to which the companies adopted continuous deployment. They also aimed to find out why continuous deployment is considered beneficial and what the obstacles are to its full adoption. The benefits mentioned the most often were the ability to get faster feedback, the ability to deploy more often to keep customers satisfied, and improved quality and productivity. Despite understanding the benefits, none of the companies adopted a fully automatic deployment pipeline. The companies also had higher continuous-deployment capability than what they practiced. In many cases, they consciously chose to not aim for full continuous deployment. Obstacles to full adoption included domain-imposed restrictions, resistance to change, customer desires, and developers' skill and confidence. <s> BIB004 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Context: Development and Operations (DevOps) is an emerging software industry movement to bridge the gap between software development and operations teams. DevOps supports frequently and reliably releasing new features and products-- thus subsuming Continuous Deployment (CD) practice. Goal: This research aims at empirically exploring the potential impact of CD practice on architecting process. Method: We carried out a case study involving interviews with 16 software practitioners. Results: We have identified (1) a range of recurring architectural challenges (i.e., highly coupled monolithic architecture, team dependencies, and ever-changing operational environments and tools) and (2) five main architectural principles (i.e., small and independent deployment units, not too much focus on reusability, aggregating logs, isolating changes, and testability inside the architecture) that should be considered when an application is (re-) architected for CD practice. This study also supports that software architecture can better support operations if an operations team is engaged at an early stage of software development for taking operational aspects into considerations. Conclusion: These findings provide evidence that software architecture plays a significant role in successfully and efficiently adopting continuous deployment. The findings contribute to establish an evidential body of knowledge about the state of the art of architecting for CD practice <s> BIB005 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> The emerging DevOps movement emphasizes development and operations staff working together as early as possible--sharing tools, processes, and practices that smooth the production path. This article is part of a theme issue on DevOps. <s> BIB006 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> DevOps is a new way of developing software that is challenging from a teaching perspective. In this paper, we outline these challenges and propose teaching methods that focus on skill acquisition and technical practices that focus on performant virtualization to overcome them. We describe central elements from our course Cloud Computing and Architecture that has been designed and executed upon these methods and practices and report our experiences and lessons learned. <s> BIB007 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> The purpose of this research is to propose what fundamental Knowledge (K), Skills (S), and Abilities (A) should be taught in web development courses, which can support not only development but also operations, namely DevOps. Instead of focusing on client side web application development using standard web technologies, we focus on server side web application development which holds many challenges due to a diversity of languages and emerging technologies. The advent of cloud computing reduced the gap between web application development and deployment/operation. Also, it brought an emerging concept, DevOps, to the web application development community. We identify what KSAs are needed to support DevOps and attempt to include them in web development courses. When the KSAs are included in web application development courses, students are well prepared for the emerging practice and computing paradigm of DevOps. <s> BIB008 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> DevOps is a predominant phenomenon in the web domain. Its two core principles emphasize collaboration between software development and operations, and the use of agile principles to manage deployment environments and their configurations. DevOps techniques, such as collaboration and behaviour-driven monitoring, have been used by web companies to facilitate continuous deployment of new functionality to customers. The techniques may also offer opportunities for continuous product improvement when adopted in the embedded systems domain. However, certain characteristics of embedded software development present obstacles for DevOps adoption, and as yet, there is no empirical evidence of its adoption in the embedded systems domain. In this study, we present the challenges for DevOps adoption in embedded systems using a multiple-case study approach with four companies. The contribution of this paper is to introduce the concept of DevOps adoption in the embedded systems domain and then to identify key challenges for the DevOps adoption. <s> BIB009 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> "Software security" often evokes negative feelings amongst software developers because this term is associated with additional programming effort, uncertainty and road blocker activity on rapid development and release cycles. The Secure DevOps movement attempts to combat the toxic environment surrounding software security by shifting the paradigm from following rules and guidelines to creatively determining solutions for tough security problems Taschner, 2015. Secure software should be focused on a proactive approach that limits the attack surface and produces reliable software. Secure DevOps developers want their software to bend but not break, which means the software absorbs attacks and continues to function. The burgeoning concepts of DevOps include a number of concepts that can be applied to increase the security of developed applications. Applying these and other DevOps principles can have a big impact on creating an environment that is resilient and secure. Specifically, this paper clearly explains how to address security concerns in the early stages of the development lifecycle and leverage that knowledge throughout the SDLC. <s> BIB010 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Emerging container technologies, such as Docker, offer unprecedented agility in developing and running applications in cloud environment especially when combined with a microservice-style architecture. However, it is often difficult to use containers to manage the cloud infrastructure, without sacrificing many benefits container offers. This paper identifies the key challenges that impede realizing the full promise of containerizing infrastructure services. Using OpenStack as a case study, we explore solutions to these challenges. Specifically, we redesign OpenStack deployment architecture to enable dynamic service registration and discovery, explore different ways to manage service state in containers, and enable containers to access the host kernel and devices. We quantify the efficiency of the container-based microservice-style DevOps compared to the VM-based approach, and study the scalability of the stateless and stateful containerized components. We also discuss limitations in our current design, and highlight open research problems that, if solved, can lead to wider adoption of containers in cloud infrastructure management. <s> BIB011 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Even though agile actively seeks collaboration from all its stakeholders, most agile projects do not extend themselves toward the operations people. To solve this problem, DevOps is introduced. DevOps is a conceptual framework for reintegrating development and operations of Information Systems, which is able to break the wall between developers and operations professionals. DevOps improves the work through a collection of principles and practices, centered around close collaboration between Development and Operations personnel. However, both sides have paid little attention to issues faced by each other. Communication gaps is a recurrent problem in agile teams that is also eminent in the relationship between developers and operations. Literature offers little research on this aspect of communication in DevOps. This position paper describes the communication practices from a distributed agile team composed of developers and operations based on communication challenges (geographical, socio-cultural, and temporal distance) and strategies (frequency, direction, modality, and content). From the results we outline possible research focus for future work, aiming to enrich the academia research on the matter as well as to help practitioners to improve their working practices. <s> BIB012 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> Many software engineering organizations around the world are adopting DevOps. One of the goals of DevOps is to foster better collaboration between development and operations personnel, in order to improve organizational efficiency. Since DevOps is lacking a common definition, there are several approaches to adopt it, and organizations largely need to determine how to apply DevOps for themselves. In this paper, we present results from a case study in which a software organization adopts DevOps. The focus of this research is to study the impact of mixing the responsibilities between development and operations engineers. We interviewed 14 employees in the organization during the study, and results indicate several benefits of the chosen approach, such as improved collaboration and trust, and smoother work flow. This comes at the cost of a number of complications, such as new sources for friction among the employees, risk for holistically sub-optimal service configurations, and more. <s> BIB013 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> DevOps and continuous development are getting popular in the software industry. Adopting these modern approaches in regulatory environments, such as medical device software, is not straightforward because of the demand for regulatory compliance. While DevOps relies on continuous deployment and integration, regulated environments require strict audits and approvals before releases. Therefore, the use of modern development approaches in regulatory environments is rare, as is the research on the topic. However, as software is more and more predominant in medical devices, modern software development approaches become attractive. This paper discusses the fit of DevOps for regulated medical device software development. We examine two related standards, IEC 62304 and IEC 82304-1, for obstacles and benefits of using DevOps for medical device software development. We found these standards to set obstacles for continuous delivery and integration. Respectively, development tools can help fulfilling the requirements of traceability and documentation of these standards. <s> BIB014 </s> A Survey of DevOps Concepts and Challenges <s> Implications for researchers <s> It's not magic, it just requires continuous, daily improvement at all levels. <s> BIB015
Based on the reviewed literature, we list DevOps-related open topics that the academic community could exploit in future research. Software Architecture: some authors explore software design in the context of DevOps, continuous delivery, and continuous deployment  BIB005 BIB006 . However, engineers may still struggle with this in practice, since achieving the desired architecture can be infeasible in a single first DevOps project. Researchers should investigate transitioning strategies to adopt practices and architectural changes. Education: DevOps adoption requires more skilled software engineers. A broader investigation is needed to explore how to teach operations skills to developers and vice-versa, as well as to introduce software operation topics in software engineering courses BIB007 BIB008 . We discuss this in more detail in Section 8.4. Embedded systems and IoT: continuously delivering new software versions in embedded systems is still a hard question BIB009 BIB001 . How to effectively adopt DevOps in an IoT context is an open research question to be investigated. Compliance: DevOps provides both benefits and obstacles to regulatory compliance BIB015  BIB014 . Research should be done to detect scenarios in which a high-frequency delivery is indeed not welcome or how one can demonstrate that automated deployment can help organizations comply with regulations, as stated by Humble BIB015 . Moreover, some practitioners advocate that engineers must have unrestricted access to production data [52,  , which can be controversial in specific environments, such as financial systems. Security: high-frequency of continuous delivery poses questions about the security in a DevOps context  BIB010 . However, DevOps can also bring benefits for security. This dual relationship generates a dichotomy that should be further investigated. Testing large-scale distributed systems: some errors in large-scale distributed systems can be hard to reproduce in testing environments  . The chaos engineering approach advocates running experiments in production [3] . This is an incipient topic in the research literature. Quantitative assessment metrics: few works use adequate quantitative metrics for assessing DevOps as is the case of the Puppet State of DevOps Reports  . The investigation and use of such quantitative metrics should be intensified in related research. Deployment approach: two automated deployment approaches are the container-based one BIB011 and the deployment scripts written in domain-specific languages such as Chef. Investigating how complementary or exclusive these approaches are can help engineers choose the best toolset. Improving interdepartmental communication: how to effectively enable interdepartmental communication is a challenge, especially for distributed organizations BIB012 . Better communication does not merely mean more communication since too much communication, and an excess of meetings can negatively impact productivity. Adoption strategies: there are still many open questions about how organizations should adopt DevOps. It is stated that DevOps adoption requires top-management support BIB003 BIB001 . Sometimes it does not happen in the first moment, and a "guerrilla" strategy can take place (i.e., acting outside the company standard procedures). Moreover, arguments to encourage DevOps adoption can differ from engineers to managers BIB002 . In such context, researchers can support DevOps adoption by providing a further investigation into balancing top-management support and guerrilla strategies; studying arguments to convince managers; providing guidelines about which DevOps-adoption strategy to choose based on organizational characteristics. Investigation involving multiple organizations: since most empirical research is conducted with data from a single organization BIB003 BIB012 BIB013 BIB013 , any conclusion from these works deserves further investigation. Surveying more companies, as done by Leppanen et al. BIB004 , can strengthen or weaken previous findings. Other research topics: Claps et al. list other technical and social challenges for continuous deployment adoption, such as product marketing, team coordination, customer adoption, feature discovery, plugin management, cross-product dependencies, and scaling CI tools BIB003 . Olsson et al. list among the challenges in continuous delivery adoption: coordination of supplier integration, business models, and difficulty in overviewing projects status BIB001 .