A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> Scaling a web applications can be easy for simple CRUD software running when you use Platform as a Service Clouds (PaaS). But if you need to deploy a complex software, with many components and a lot users, you will need have a mix of cloud services in PaaS, SaaS and IaaS layers. You will also need knowledge in architecture patterns to make all these software components communicate accordingly. In this article, we share our experience of using cloud services to scale a web application. We show usage examples of load balancing, session sharing, e-mail delivery, asynchronous processing, logs processing, monitoring, continuous deployment, realtime user monitoring (RUM). These are a mixture of development and system operations (DevOps) that improved our application availability, scalability and performance. <s> BIB001 </s> A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> As part of a Finnish research program, researchers interviewed 15 information and communications technology companies to determine the extent to which the companies adopted continuous deployment. They also aimed to find out why continuous deployment is considered beneficial and what the obstacles are to its full adoption. The benefits mentioned the most often were the ability to get faster feedback, the ability to deploy more often to keep customers satisfied, and improved quality and productivity. Despite understanding the benefits, none of the companies adopted a fully automatic deployment pipeline. The companies also had higher continuous-deployment capability than what they practiced. In many cases, they consciously chose to not aim for full continuous deployment. Obstacles to full adoption included domain-imposed restrictions, resistance to change, customer desires, and developers' skill and confidence. <s> BIB002 </s> A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> Context: Development and Operations (DevOps) is an emerging software industry movement to bridge the gap between software development and operations teams. DevOps supports frequently and reliably releasing new features and products-- thus subsuming Continuous Deployment (CD) practice. Goal: This research aims at empirically exploring the potential impact of CD practice on architecting process. Method: We carried out a case study involving interviews with 16 software practitioners. Results: We have identified (1) a range of recurring architectural challenges (i.e., highly coupled monolithic architecture, team dependencies, and ever-changing operational environments and tools) and (2) five main architectural principles (i.e., small and independent deployment units, not too much focus on reusability, aggregating logs, isolating changes, and testability inside the architecture) that should be considered when an application is (re-) architected for CD practice. This study also supports that software architecture can better support operations if an operations team is engaged at an early stage of software development for taking operational aspects into considerations. Conclusion: These findings provide evidence that software architecture plays a significant role in successfully and efficiently adopting continuous deployment. The findings contribute to establish an evidential body of knowledge about the state of the art of architecting for CD practice <s> BIB003 </s> A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> When DevOps started gaining momentum in the software industry, one of the first service-based architectural styles to be introduced, be applied in practice, and become popular was microservices. Migrating monolithic architectures to cloud-native architectures such as microservices reaps many benefits, such as adaptability to technological changes and independent resource management for different system components. This article reports on experiences and lessons learned during incremental migration and architectural refactoring of a commercial MBaaS (mobile back end as a service) to microservices. It explains how adopting DevOps facilitated a smooth migration. Furthermore, the researchers transformed their experiences in different projects into reusable migration practices, resulting in microservices migration patterns. This article is part of a theme issue on DevOps. The Web extra at https://youtu.be/MF3-dKTCQ88 is an audio recording of Brian Brannon speaking with author Pooyan Jamshidi and James Lewis, principal engineer at ThoughtWorks, about DevOps and microservices architecture. <s> BIB004 </s> A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> Wotif Group used DevOps principles to recover from the downward spiral of manual release activity that many IT departments face. Its approach involved the idea of "making it easy to do the right thing." By defining the right thing (deployment standards) for development and operations teams and making it easy to adopt, Wotif drastically improved the average release cycle time. This article is part of a theme issue on DevOps. <s> BIB005 </s> A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> Whether it's in Java, .NET, or Ruby on Rails, getting your application ready to ship is only half the battle. Did you design your system to survive a sudden rush of visitors from Digg or Slashdot? Or an influx of real world customers from 100 different countries? Are you ready for a world filled with flakey networks, tangled databases, and impatient users? If you're a developer and don't want to be on call at 3AM for the rest of your life, this book will help. In "Release It!", Michael T. Nygard shows you how to design and architect your application for the harsh realities it will face. You'll learn how to design your application for maximum uptime, performance, and return on investment. Mike explains that many problems with systems today start with the design: "It's disconnected from the real world. It's the same as cars designed solely in the cool comfort of the lab - they look great in models and CAD systems, but don't work well in the real world. You want a car designed by somebody who knows that oil changes are always 3,000 miles late; that the tires must work just as well on the last sixteenth of an inch of tread as on the first; and that you will certainly, at some point, stomp on the brakes while you're holding an Egg McMuffin in one hand and a cell phone in the other." With a combination of case studies and practical advice, Patterns to follow and Anti-Patterns to avoid, "Release It!" will help you manage the pitfalls that cost companies huge amounts of time and money each year. <s> BIB006 </s> A Survey of DevOps Concepts and Challenges <s> How to re-design systems toward continuous delivery <s> It's not magic, it just requires continuous, daily improvement at all levels. <s> BIB007
Highly coupled monolithic architectures are obstacles to effective continuous delivery. Complex dependency management of software components and teams is imposed to the deployment pipeline BIB003 . An essential principle for successfully adopting and implementing continuous delivery is an architecture composed of small and independently deployable units, also called microservices BIB003 . In such architectural style, services interact through the network and are built around business capabilities [94] . Conceiving of an application architecture with loosely coupled and well-encapsulated microservices guarantees two architectural attributes required by continuous delivery: testability and deployability BIB007 . If each microservice has its test suite, this reduces the blocking of deployment due to long-running tests BIB002 . Whereas some authors say microservices facilitate effective implementation of DevOps BIB004 , others say microservices require DevOps [17] , since deployment automation minimizes the overhead to manage a significant number of microservices. However, adopting microservices comes with several challenges. First, there is heterogeneity in non-functional patterns such as "startup scripts, configuration files, administration endpoints, and logging locations" BIB005 . Technological heterogeneity can be a productivity barrier for newcomers in the team. Second, microservices must be deployed to production with the same set of versions used for integration tests  . These challenges can be overcome by adopting a minimum set of microservices standards across the organization BIB005 . Common microservice management patterns associated with DevOps are: one deployment pipeline per microservice; log aggregator; service registry; correlation ID's  ; and segregation of source code, configuration, and environment specification BIB004 . Other complimentary patterns are strangler application [72]  BIB007 , load balancer, consumer-driven contracts BIB004 , circuit breaker BIB006 , backwards compatibility, and versioned APIs BIB007 . However, the last one is controversial, and some communities usually do not recommend microservice versioning BIB004 . In highly regulated environments, the proper use of microservices can constrain unfriendly regulations to specific system modules and people BIB007 . Additionally, the usage of a platform-as-aservice (PaaS) can automate much of the compliance checking, a model used, for example, by the U.S. federal government BIB007 . PaaS is also recommended for its operational simplicity, like other cloud services, such as storage services, asynchronous processing with queues, email delivery, and real-time user monitoring BIB001 . Finally, although reusability is a historical goal of software engineering  , Shahin et al. recommend engineers not to focus too much on it BIB003 . It brings coupling, which is a huge bottleneck to continuous delivery BIB003 . Therefore, each team should discuss the trade-off between reusability and independence from other components, services, and teams.