A Survey of DevOps Concepts and Challenges <s> Tools for deployment automation <s> Scaling a web applications can be easy for simple CRUD software running when you use Platform as a Service Clouds (PaaS). But if you need to deploy a complex software, with many components and a lot users, you will need have a mix of cloud services in PaaS, SaaS and IaaS layers. You will also need knowledge in architecture patterns to make all these software components communicate accordingly. In this article, we share our experience of using cloud services to scale a web application. We show usage examples of load balancing, session sharing, e-mail delivery, asynchronous processing, logs processing, monitoring, continuous deployment, realtime user monitoring (RUM). These are a mixture of development and system operations (DevOps) that improved our application availability, scalability and performance. <s> BIB001 </s> A Survey of DevOps Concepts and Challenges <s> Tools for deployment automation <s> Continuous delivery and deployment are dramatically shortening release cycles from months to hours. Cloud applications with high-frequency releases often rely heavily on automated tools and cloud infrastructure APIs to deploy new software versions. The authors report on reliability issues and how these tools and APIs contribute to them. They also analyze the trade-offs between using heavily baked and lightly baked virtual-image approaches, on the basis of experiments with Amazon Web Service OpsWorks APIs and the Chef configuration management tool. Finally, they propose error-handling practices for continuous-delivery facilities. <s> BIB002 </s> A Survey of DevOps Concepts and Challenges <s> Tools for deployment automation <s> Virtualization, cloud, containers, server automation, and software-defined networking are meant to simplify IT operations. But many organizations adopting these technologies have found that it only leads to a faster-growing sprawl of unmanageable systems. This is where infrastructure as code can help. With this practical guide, author Kief Morris of Thought Works shows you how to effectively use principles, practices, and patterns pioneered through the DevOps movement to manage cloud age infrastructure. Ideal for system administrators, infrastructure engineers, team leads, and architects, this book demonstrates various tools, techniques, and patterns you can use to implement infrastructure as code. In three parts, youll learn about the platforms and tooling involved in creating and configuring infrastructure elements, patterns for using these tools, and practices for making infrastructure as code work in your environment. Examine the pitfalls that organizations fall into when adopting the new generation of infrastructure technologies Understand the capabilities and service models of dynamic infrastructure platforms Learn about tools that provide, provision, and configure core infrastructure resources Explore services and tools for managing a dynamic infrastructureLearn specific patterns and practices for provisioning servers, building server templates, and updating running servers <s> BIB003 </s> A Survey of DevOps Concepts and Challenges <s> Tools for deployment automation <s> Emerging container technologies, such as Docker, offer unprecedented agility in developing and running applications in cloud environment especially when combined with a microservice-style architecture. However, it is often difficult to use containers to manage the cloud infrastructure, without sacrificing many benefits container offers. This paper identifies the key challenges that impede realizing the full promise of containerizing infrastructure services. Using OpenStack as a case study, we explore solutions to these challenges. Specifically, we redesign OpenStack deployment architecture to enable dynamic service registration and discovery, explore different ways to manage service state in containers, and enable containers to access the host kernel and devices. We quantify the efficiency of the container-based microservice-style DevOps compared to the VM-based approach, and study the scalability of the stateless and stateful containerized components. We also discuss limitations in our current design, and highlight open research problems that, if solved, can lead to wider adoption of containers in cloud infrastructure management. <s> BIB004 </s> A Survey of DevOps Concepts and Challenges <s> Tools for deployment automation <s> Many software engineering organizations around the world are adopting DevOps. One of the goals of DevOps is to foster better collaboration between development and operations personnel, in order to improve organizational efficiency. Since DevOps is lacking a common definition, there are several approaches to adopt it, and organizations largely need to determine how to apply DevOps for themselves. In this paper, we present results from a case study in which a software organization adopts DevOps. The focus of this research is to study the impact of mixing the responsibilities between development and operations engineers. We interviewed 14 employees in the organization during the study, and results indicate several benefits of the chosen approach, such as improved collaboration and trust, and smoother work flow. This comes at the cost of a number of complications, such as new sources for friction among the employees, risk for holistically sub-optimal service configurations, and more. <s> BIB005
Deployment automation tools are employed in the deployment stages of the pipeline to make the continuous delivery process viable. They enable frequent and reliable deployment processes, as well as other concepts related to the delivery category. Although the use of deployment automation tools is a joint effort between developers and operators, the primary mission of continuous delivery is putting the deployment schedule under business control. Every automated deployment approach relies on the concept of "infrastructure as a code." It requires engineers to specify servers, networking, and other infrastructure elements in files modeled after software code BIB003 . In this model, deployment and infrastructure definitions are shared among developers and operators, allowing effective cooperation between them. Automated deployment can encompass not only the deployment of the package to the production environment but also the provisioning of the target environment. Such provisioning is usually performed in cloud environments BIB001 , such as Amazon Web Services, Google Cloud, and Azure. These platforms deliver a vast amount of infrastructure services via Infrastructure as a Service (IaaS) model, like launching virtual machines, databases, queues, etc. The creation of all these resources can also be orchestrated, in Amazon's platform, through the usage of AWS Cloud Formation. Operators and developers can share the task of using IaaS services. It is also possible to use a Platform as a Service (PaaS), such as Heroku. In the PaaS approach, the platform is responsible for the deployment, and developers do not know the underlying virtual infrastructure. Serverless services -a new trend in cloud computing -aim to abstract servers for developers, which is close to PaaS; the main differences are the scaling transparency and the charge for computing usage  . These models suggest there is no need for specialized operations teams BIB001 ; or, at least, that fewer people are required for operations  . If using IaaS, the environment provisioning is followed by the execution of scripts that effectively install the application in the target environment. A deployment script can be written using Shell Script, but configuration management tools such as Chef and Puppet offer advantages by leveraging operation system portability and idempotence mechanisms, which are difficult to achieve with Shell Script. An example of language construct of Chef that leads to portability is using the "package" resource, which is resolved to a concrete package manager, such as Apt, only at execution time. An example of an idempotent mechanism of Puppet in the "service" resource is declaring the desired final state of the service as "running" rather than writing a command to start the service  . Another alternative for deployment is containerization, primarily implemented by Docker. Docker containers resemble virtual machines. The main difference is that they are more lightweight BIB004 , considering they share the kernel of the host. Docker and related tools, such as Docker Compose, Kubernetes, and Rancher, allow the specification of containers and their dependency relationships. These specifications generate container images in the build stage, which are later instantiated in the target environment. Docker has been used not only for deploying applications but also for deploying the underlying infrastructure BIB004 . Containerization could be seen as complementary to the deployment script strategy (done by Chef or Puppet). However, in practice, these strategies seem to compete. A Chef script is executed continuously on the target node, and its success depends on the previous target node state. On the other hand, in containerization the whole containerized environment is generated in the build, so the environment is destroyed and rebuilt at each new software version. When compared to the Chef strategy, Docker yields faster and more reliable deployment BIB004 , but at the expense of bigger builds. Complimentary usage of configuration management and containers entails the setup of Docker environment and update of the operating system software, like SSH. Another usage is to manage configuration that varies across diverse environments (testing, staging, production). Such a configuration cannot be embedded in container images since the same image must be deployed in every environment. Nonetheless, the debate about configuration management versus containers is not fully explored in the literature. Among practitioners, although it is possible to find support for a complementary usage of both  , there is a trend to favor containers over configuration management tools    . Zhu et al. compare the provisioning reliability of two approaches: using "lightly baked images" versus "heavily baked images" BIB002 . In the lightly baked approach, a virtual machine image is instantiated, and Chef is executed to install the application, whereas in the heavily baked approach the image already contains the entire application. This heavily baked approach is similar to the container strategy, but using virtual machine technology. The authors conclude that deployment using lightly baked images is less reliable since it involves more external resources during deployment, which tends to increase errors and delays. As the application evolves, the database structure evolves as well. Traditionally, "database administrators" maintain the database structure. The adoption of emergent design  and the need for frequent releases have encouraged the use of database migrations tools such as Flyway, which controls the automated application of schema updates across different environments, thus enabling developers to manage database structure themselves. In this context, one must rethink the traditional role of database administrators. Developers and operators share these tools for deployment. The main risk is the absence of clarity of responsibilities, which can cause friction in organizations during DevOps adoption BIB005 . Developers believe they do not control their application and feel they are doing someone else's job. Additionally, operators may judge developers as unable to perform some tasks or operate some tools.