A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Software development projects can be fun, productive, and even daring. Yet they can consistently deliver value to a business and remain under control.Extreme Programming (XP) was conceived and developed to address the specific needs of software development conducted by small teams in the face of vague and changing requirements. This new lightweight methodology challenges many conventional tenets, including the long-held assumption that the cost of changing a piece of software necessarily rises dramatically over the course of time. XP recognizes that projects have to work to achieve this reduction in cost and exploit the savings once they have been earned.Fundamentals of XP include: Distinguishing between the decisions to be made by business interests and those to be made by project stakeholders. Writing unit tests before programming and keeping all of the tests running at all times. Integrating and testing the whole system--several times a day. Producing all software in pairs, two programmers at one screen. Starting projects with a simple design that constantly evolves to add needed flexibility and remove unneeded complexity. Putting a minimal system into production quickly and growing it in whatever directions prove most valuable.Why is XP so controversial? Some sacred cows don't make the cut in XP: Don't force team members to specialize and become analysts, architects, programmers, testers, and integrators--every XP programmer participates in all of these critical activities every day. Don't conduct complete up-front analysis and design--an XP project starts with a quick analysis of the entire system, and XP programmers continue to make analysis and design decisions throughout development. Develop infrastructure and frameworks as you develop your application, not up-front--delivering business value is the heartbeat that drives XP projects. Don't write and maintain implementation documentation--communication in XP projects occurs face-to-face, or through efficient tests and carefully written code.You may love XP, or you may hate it, but Extreme Programming Explained will force you to take a fresh look at how you develop software. 0201616416B04062001 <s> BIB001 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> The system-to-administrator ratio is commonly used as a rough metric to understand administrative costs in high-scale services. With smaller, less automated services this ratio can be as low as 2:1, whereas on industry leading, highly automated services, we've seen ratios as high as 2, 500:1. Within Microsoft services, Autopilot [1] is often cited as the magic behind the success of the Windows Live Search team in achieving high system-to-administrator ratios. While autoadministration is important, the most important factor is actually the service itself. Is the service efficient to automate? Is it what we refer to more generally as operations-friendly? Services that are operations-friendly require little human intervention, and both detect and recover from all but the most obscure failures without administrative intervention. This paper summarizes the best practices accumulated over many years in scaling some of the largest services at MSN and Windows Live. <s> BIB002 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Scaling a web applications can be easy for simple CRUD software running when you use Platform as a Service Clouds (PaaS). But if you need to deploy a complex software, with many components and a lot users, you will need have a mix of cloud services in PaaS, SaaS and IaaS layers. You will also need knowledge in architecture patterns to make all these software components communicate accordingly. In this article, we share our experience of using cloud services to scale a web application. We show usage examples of load balancing, session sharing, e-mail delivery, asynchronous processing, logs processing, monitoring, continuous deployment, realtime user monitoring (RUM). These are a mixture of development and system operations (DevOps) that improved our application availability, scalability and performance. <s> BIB003 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Internet companies such as Facebook operate in a "perpetual development" mindset. This means that the website continues to undergo development with no predefined final objective, and that new developments are deployed so that users can enjoy them as soon as they're ready. To support this, Facebook uses both technical approaches such as peer review and extensive automated testing, and a culture of personal responsibility. <s> BIB004 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Abstract Context Continuous Deployment (CD) is an emerging software development process with organisations such as Facebook, Microsoft, and IBM successfully implementing and using the process. The CD process aims to immediately deploy software to customers as soon as new code is developed, and can result in a number of benefits for organisations, such as: new business opportunities, reduced risk for each release, and prevent development of wasted software. There is little academic literature on the challenges organisations face when adopting the CD process, however there are many anecdotal challenges that organisations have voiced on their online blogs. Objective The aim of this research is to examine the challenges faced by organisations when adopting CD as well as the strategies to mitigate these challenges. Method An explorative case study technique that involves in-depth interviews with software practitioners in an organisation that has adopted CD was conducted to identify these challenges. Results This study found a total of 20 technical and social adoption challenges that organisations may face when adopting the CD process. The results are discussed to gain a deeper understanding of the strategies employed by organisations to mitigate the impacts of these challenges. Conclusion While a number of individual technical and social adoption challenges were uncovered by the case study in this research, most challenges were not faced in isolation. The severity of these challenges were reduced by a number of mitigation strategies employed by the case study organisation. It is concluded that organisations need to be well prepared to handle technical and social adoption challenges with their existing expertise, processes and tools before adopting the CD process. For practitioners, knowing how to address the challenges an organisation may face when adopting the CD process provides a level of awareness that they previously may not have had. <s> BIB005 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Continuous delivery (CD) has emerged as an auspicious alternative to traditional release engineering, promising to provide the capability to release valuable software continuously to customers. Paddy Power has been implementing CD for the past two years. This article explains why Paddy Power decided to adopt CD, describes the resulting CD capability, and reports the huge benefits and challenges involved. These experiences can provide fellow practitioners with insights for their adoption of CD, and the identified challenges can provide researchers valuable input for developing their research agendas. <s> BIB006 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Many software engineering organizations around the world are adopting DevOps. One of the goals of DevOps is to foster better collaboration between development and operations personnel, in order to improve organizational efficiency. Since DevOps is lacking a common definition, there are several approaches to adopt it, and organizations largely need to determine how to apply DevOps for themselves. In this paper, we present results from a case study in which a software organization adopts DevOps. The focus of this research is to study the impact of mixing the responsibilities between development and operations engineers. We interviewed 14 employees in the organization during the study, and results indicate several benefits of the chosen approach, such as improved collaboration and trust, and smoother work flow. This comes at the cost of a number of complications, such as new sources for friction among the employees, risk for holistically sub-optimal service configurations, and more. <s> BIB007 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Even though agile actively seeks collaboration from all its stakeholders, most agile projects do not extend themselves toward the operations people. To solve this problem, DevOps is introduced. DevOps is a conceptual framework for reintegrating development and operations of Information Systems, which is able to break the wall between developers and operations professionals. DevOps improves the work through a collection of principles and practices, centered around close collaboration between Development and Operations personnel. However, both sides have paid little attention to issues faced by each other. Communication gaps is a recurrent problem in agile teams that is also eminent in the relationship between developers and operations. Literature offers little research on this aspect of communication in DevOps. This position paper describes the communication practices from a distributed agile team composed of developers and operations based on communication challenges (geographical, socio-cultural, and temporal distance) and strategies (frequency, direction, modality, and content). From the results we outline possible research focus for future work, aiming to enrich the academia research on the matter as well as to help practitioners to improve their working practices. <s> BIB008 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Context: Development and Operations (DevOps) is an emerging software industry movement to bridge the gap between software development and operations teams. DevOps supports frequently and reliably releasing new features and products-- thus subsuming Continuous Deployment (CD) practice. Goal: This research aims at empirically exploring the potential impact of CD practice on architecting process. Method: We carried out a case study involving interviews with 16 software practitioners. Results: We have identified (1) a range of recurring architectural challenges (i.e., highly coupled monolithic architecture, team dependencies, and ever-changing operational environments and tools) and (2) five main architectural principles (i.e., small and independent deployment units, not too much focus on reusability, aggregating logs, isolating changes, and testability inside the architecture) that should be considered when an application is (re-) architected for CD practice. This study also supports that software architecture can better support operations if an operations team is engaged at an early stage of software development for taking operational aspects into considerations. Conclusion: These findings provide evidence that software architecture plays a significant role in successfully and efficiently adopting continuous deployment. The findings contribute to establish an evidential body of knowledge about the state of the art of architecting for CD practice <s> BIB009 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> When DevOps started gaining momentum in the software industry, one of the first service-based architectural styles to be introduced, be applied in practice, and become popular was microservices. Migrating monolithic architectures to cloud-native architectures such as microservices reaps many benefits, such as adaptability to technological changes and independent resource management for different system components. This article reports on experiences and lessons learned during incremental migration and architectural refactoring of a commercial MBaaS (mobile back end as a service) to microservices. It explains how adopting DevOps facilitated a smooth migration. Furthermore, the researchers transformed their experiences in different projects into reusable migration practices, resulting in microservices migration patterns. This article is part of a theme issue on DevOps. The Web extra at https://youtu.be/MF3-dKTCQ88 is an audio recording of Brian Brannon speaking with author Pooyan Jamshidi and James Lewis, principal engineer at ThoughtWorks, about DevOps and microservices architecture. <s> BIB010 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> The overwhelming majority of a software systems lifespan is spent in use, not in design or implementation. So, why does conventional wisdom insist that software engineers focus primarily on the design and development of large-scale computing systems? In this collection of essays and articles, key members of Googles Site Reliability Team explain how and why their commitment to the entire lifecycle has enabled the company to successfully build, deploy, monitor, and maintain some of the largest software systems in the world. Youll learn the principles and practices that enable Google engineers to make systems more scalable, reliable, and efficientlessons directly applicable to your organization. This book is divided into four sections: Introduction Learn what site reliability engineering is and why it differs from conventional IT industry practices Principles Examine the patterns, behaviors, and areas of concern that influence the work of a site reliability engineer (SRE) Practices Understand the theory and practice of an SREs day-to-day work: building and operating large distributed computing systems Management Explore Google's best practices for training, communication, and meetings that your organization can use <s> BIB011 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> The DevOps phenomenon is gaining popularity through its ability to support continuous value delivery and ready accommodation of change. However, given the relative immaturity and general confusion about DevOps, a common view of expectations from a DevOps role is lacking. Through investigation of online job advertisements, combined with interviews, we identified key Knowledge Areas, Skills and Capabilities for a DevOps role and their relative importance in New Zealand's job market. Our analysis also revealed the global dimensions and the emerging nature of the DevOps role in GSE projects. This research adds a small advanced economy (New Zealand) perspective to the literature on DevOps job advertisements and should be of value to employers, job seekers, researchers as well educators and policy makers. <s> BIB012 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> Many software services are currently created using DevOps, where developers and operations personnel are more tightly integrated. The DevOps paradigm enables shorter development cycles, but increased speed has raised concerns over whether security issues may be overlooked. However, perfect security is never achievable, and in addition to the proactive software security efforts, we also need a reactive effort to handle flaws and bugs that are not discovered before they are used in an attack. In this paper we explore how focus on incident management and collaboration with developers can contribute to improved software security. <s> BIB013 </s> A Survey of DevOps Concepts and Challenges <s> How to deploy DevOps in an organization <s> For many software development teams, the first aspects that come to mind regarding continuous delivery (CD) are the operational challenges and competitive benefits. In the authors’ experience, CD was much more: it was a survival technique. This article presents how and why they applied CD in a large governmental project for the development of a collaborative development environment. They share the challenges they faced and the strategies they used to overcome them. The article concludes with a set of lessons learned that can be valuable for readers in a variety of situations. This article is part of a special issue on release engineering. <s> BIB014
A hard question not yet fully answered by the literature is how -perhaps whether -an organization should be restructured to adopt DevOps. We believe the literature is incomplete and even contradictory regarding this subject. We discuss a few studies that have evaluated this matter. The seminal paper of Humble and Molesky about DevOps presents three scenarios. First, preserving the structures of development and operations departments, DevOps is led by human collaboration among developers and operators, with operators attending agile ceremonies and developers contributing to incident solving  BIB013 . Alternatively, product teams, also called crossfunctional teams, effectively incorporate operators. Finally, in a third scenario, one product team is composed of only operators offering support services (continuous integration, monitoring services) to the entire organization. Similarly, Nybom et al. also present three distinct scenarios to DevOps adoption BIB007 : i) collaboration among development and operations departments; ii) cross-functional teams; and iii) creation of "DevOps teams. " These approaches are summarized in the Table 5 and discussed in detail in the following paragraphs. In the first approach, the responsibilities of developers overlap with operators BIB005 BIB008 BIB007 BIB009 . High risk for adopting this strategy occurs when new responsibilities do not become evident in the organization BIB005 BIB007 , which could lead to frictions among developers and operators BIB006 . This is especially true when delivery automation is not prioritized BIB007 . Cross-functional teams resemble the "whole team" practice from XP BIB001 , and seems to prevail in literature BIB010 BIB003 BIB004  . This structure appeals to "T-shaped" professionals, who have expertise in few fields and basic skills in multiple correlated areas   . In this model, at least one team member must master operations skills. From this perspective, the so-called "DevOps engineer" BIB012 , also called the "full stack engineer" BIB012 , is known as a developer with operations skills. One could wonder whether it makes sense to talk about developers and operators collaboration in cross-functional teams context. Adopting cross-functional teams may just be a matter of moving operators to product teams. However, this is not trivial, considering that large organizations usually have only a small pool of operators for several development teams BIB007 . Therefore, companies must hire people with both development and operations skills or train some of their developers in operations skills. Nevertheless, pressuring developers to learn operations can be ineffective, as they are already overwhelmed with other skills they need to learn. Transforming development teams into product teams has still another major cultural shift, as developers become responsible for 24/7 service support  BIB009 . Even when services should be designed to remain available without human interaction BIB002 . An extra challenge for cross-functional teams is guaranteeing that specialists interact with their peer groups to share novelties in the field  . Spotify, for example, achieves this through inter-team "guilds, " which are communities of common interest within the organization  . Assigning a "DevOps team" as a bridge between developers and operators BIB007 has become a trend  . However, it is not clear what precisely "DevOps teams" are, and how they differ from regular operations teams BIB007  . This approach is criticized by Humble, who considers that potentially creating a new silo is not the best policy to handle the DevOps principle of breaking down walls between silos  . Skelton and Pais present some variations, called "DevOps topologies, " and also DevOps anti-patterns  . They argue that, although the DevOps team silo is an antipattern, it is acceptable when it is temporary and focused on sharing development practices to operations staff and operations concerns to developers. Siqueira et al. report how a DevOps team with senior developers and rotating roles among members was a key decision to construct and maintain a continuous delivery process BIB014 . There is yet the strategy adopted by Google, called Site Reliability Engineering (SRE) BIB011 , which involves the evolution of the operations engineer role. In addition to product teams, Google has SRE teams responsible for product reliability engineering, what includes performance, availability, monitoring, and emergency response. SRE teams have an upper limit of using 50 percent of their time on operational tasks because they must allocate at least 50 percent of their time increasing product reliability through engineering and development activities. One should also question the impact of deployment automation on operators' roles. Chen reported that, after continuous delivery adoption, operations engineers only needed to click a button to release new versions BIB006 . One should question the role of an engineer in such a context. Moreover, if software updates are adequately broken down into small increments, and delivery automation turns deployment in a "non-event"  , practices such as "development and operations teams should celebrate successful releases together"  can be perceived as contradictory. Therefore, if operations provide support services (continuous integration, monitoring)  and product reliability is appropriately designed, product teams can easily deploy and operate their services. Thus, one can even replace operations with third-party cloud services, as done by Cukier BIB003 , and achieve an organizational structure known as "NoOps". Some blog posts promote NoOps concepts, but the literature has not yet covered this subject. One possible definition is: "NoOps (no operations) is the concept that an IT environment can become so automated and abstracted from the underlying infrastructure that there is no need for a dedicated team to manage software in-house"  . Cross-functional teams can embrace NoOps, with a particular focus on employing cloud technology that automates much of the operational work, rather than reallocating operators or teaching developers all the operational work.