<commit_before>---
title: Overcoming YAML Pain in Continuous Integration
key: yaml-hell-in-devops
tags:
- .NET
- AppVeyor
- Azure Pipelines
- Continuous Integration
- DevOps
- GitHub Actions
- Kotlin
- NUKE
- TeamCity
mode: immersive
header:
  theme: dark
article_header:
  type: overlay
  theme: dark
  background_color: '#203028'
  background_image:
    gradient: 'linear-gradient(135deg, rgba(34, 0, 170 , .2), rgba(139, 34, 139, .2))'
    src: assets/images/2020-02-04-yaml-hell-in-devops/cover.jpg
---

<script>
{%- include scripts/lib/swiper.js -%}
var SOURCES = window.TEXT_VARIABLES.sources;
window.Lazyload.js(SOURCES.jquery, function() {
  $('.swiper-demo').swiper();
});
</script>

<style>
  .swiper-demo {
    margin: 30px auto;
  }
  .swiper-demo .swiper__slide {
    display: flex;
    align-items: center;
    justify-content: center;
    font-size: 3rem;
    color: #fff;
  }
  .swiper-demo .swiper__slide:nth-child(even) {
    background-color: #ff69b4;
  }
  .swiper-demo .swiper__slide:nth-child(odd) {
    background-color: #2593fc;
  }
  .swiper-demo--dark .swiper__slide:nth-child(even) {
    background-color: #312;
  }
  .swiper-demo--dark .swiper__slide:nth-child(odd) {
    background-color: #123;
  }
  .swiper-demo--image .swiper__slide:nth-child(n) {
    background-color: #000;
  }
</style>

***TL;DR: YAML can be a wolf in sheep’s clothing when used for CI infrastructure. Other alternatives include moving to build systems like CAKE/FAKE, or using TeamCity and its Kotlin DSL. Both approaches have their unique pros and cons. NUKE, of which I'm the author, provides a solution that combines the power of both - flexibility of a build system, and CI specific features like parallelization and build queue optimizations.***

If you dive into the DevOps world, chances are high you **meet YAML around the next corner**. For some tools, like Docker and Kubernetes, I think it's a good match. However, for CI infrastructure it often becomes a nightmare, and actually, I'm not alone having such feelings. Recently, a tweet of <NAME> started a debate about [YAML in DevOps](https://twitter.com/csharpfritz/status/1207431413341081601), to which the general agreement can be summarized as:

<div class="tweet" tweetID="1207433967097520129">Welcome the world of YAML pain</div>

Many of us only use YAML reluctantly. Personally, I would even say that the idea of _Configuration as Code_ is a lie, because it doesn't feel like coding (more about this in the next section). Yet, almost every CI/CD service out there is YAML-first:

- [Azure Pipelines](https://docs.microsoft.com/en-us/azure/devops/pipelines)
- [Bitrise](https://devcenter.bitrise.io/)
- [TravisCI](https://docs.travis-ci.com/)
- [GitHub Actions](https://help.github.com/en/actions)
- [AppVeyor](https://www.appveyor.com/docs/)

Some try to stay clear from YAML, only to replace it with other suboptimal solutions. For instance, Jenkins uses a Groovy-flavoured configuration files, which isn't really great either.

## What's wrong with YAML

While I see how YAML configuration can be attractive, I truly believe that for CI/CD purpose, this is only because the sample pipelines in talks and blog posts are often just that – samples. [YAML is in disrepute](https://noyaml.com/) for several reasons. Some of the most important ones, in particular for CI/CD, are:

- **It imposes long feedback loops.** Typically, the only way to test your configuration, is to commit your changes to the repository. Then the CI servers needs to pick up changes, and finally you might need to wait until the agent is available to trigger a new build. This adds up to a lot of time, especially given the following pitfalls.
- **It's error-prone.** We might indent too much or too little, mistype a well-known property, or forget to escape properly without even knowing. YAML is almost always valid, and there is no proper syntax highlighting. There are schema files that enable rudimentary code completion, but as a C# developer, this still feels clunky.
- **It's not refactoring-safe.** This is a matter of tooling again. Whenever we're dealing with IDs and their references, the best choice you have is _Search & Replace_. This should only be the last resort.<commit_msg>Update title and one more tweet<commit_after>---
title: Overcoming YAML Pain in CI/CD
key: yaml-hell-in-devops
tags:
- .NET
- AppVeyor
- Azure Pipelines
- Continuous Integration
- DevOps
- GitHub Actions
- Kotlin
- NUKE
- TeamCity
mode: immersive
header:
  theme: dark
article_header:
  type: overlay
  theme: dark
  background_color: '#203028'
  background_image:
    gradient: 'linear-gradient(135deg, rgba(34, 0, 170 , .2), rgba(139, 34, 139, .2))'
    src: assets/images/2020-02-04-yaml-hell-in-devops/cover.jpg
---

<script>
{%- include scripts/lib/swiper.js -%}
var SOURCES = window.TEXT_VARIABLES.sources;
window.Lazyload.js(SOURCES.jquery, function() {
  $('.swiper-demo').swiper();
});
</script>

<style>
  .swiper-demo {
    margin: 30px auto;
  }
  .swiper-demo .swiper__slide {
    display: flex;
    align-items: center;
    justify-content: center;
    font-size: 3rem;
    color: #fff;
  }
  .swiper-demo .swiper__slide:nth-child(even) {
    background-color: #ff69b4;
  }
  .swiper-demo .swiper__slide:nth-child(odd) {
    background-color: #2593fc;
  }
  .swiper-demo--dark .swiper__slide:nth-child(even) {
    background-color: #312;
  }
  .swiper-demo--dark .swiper__slide:nth-child(odd) {
    background-color: #123;
  }
  .swiper-demo--image .swiper__slide:nth-child(n) {
    background-color: #000;
  }
</style>

***TL;DR: YAML can be a wolf in sheep’s clothing when used for CI infrastructure. Other alternatives include moving to build systems like CAKE/FAKE, or using TeamCity and its Kotlin DSL. Both approaches have their unique pros and cons. NUKE, of which I'm the author, provides a solution that combines the power of both - flexibility of a build system, and CI specific features like parallelization and build queue optimizations.***

If you dive into the DevOps world, chances are high you **meet YAML around the next corner**. For some tools, like Docker and Kubernetes, I think it's a good match. However, for CI infrastructure it often becomes a nightmare, and actually, I'm not alone having such feelings. Recently, a tweet of <NAME> started a debate about [YAML in DevOps](https://twitter.com/csharpfritz/status/1207431413341081601), to which the general agreement can be summarized as:

<div class="tweet" tweetID="1207433967097520129">Welcome the world of YAML pain</div>

Here's another one:

<div class="tweet" tweetID="1237145762229673984">YAML CAN KISS MY &SS</div>

Many of us only use YAML reluctantly. Personally, I would even say that the idea of _Configuration as Code_ is a lie, because it doesn't feel like coding (more about this in the next section). Yet, almost every CI/CD service out there is YAML-first:

- [Azure Pipelines](https://docs.microsoft.com/en-us/azure/devops/pipelines)
- [Bitrise](https://devcenter.bitrise.io/)
- [TravisCI](https://docs.travis-ci.com/)
- [GitHub Actions](https://help.github.com/en/actions)
- [AppVeyor](https://www.appveyor.com/docs/)

Some try to stay clear from YAML, only to replace it with other suboptimal solutions. For instance, Jenkins uses a Groovy-flavoured configuration files, which isn't really great either.

## What's wrong with YAML

While I see how YAML configuration can be attractive, I truly believe that for CI/CD purpose, this is only because the sample pipelines in talks and blog posts are often just that – samples. [YAML is in disrepute](https://noyaml.com/) for several reasons. Some of the most important ones, in particular for CI/CD, are:

- **It imposes long feedback loops.** Typically, the only way to test your configuration, is to commit your changes to the repository. Then the CI servers needs to pick up changes, and finally you might need to wait until the agent is available to trigger a new build. This adds up to a lot of time, especially given the following pitfalls.
- **It's error-prone.** We might indent too much or too little, mistype a well-known property, or forget to escape properly without even knowing. YAML is almost always valid, and there is no proper syntax highlighting. There are schema files that enable rudimentary code completion, but as a C# developer, this still feels clunky.
- **It's not refactoring-safe.** This is a matter of tooling again. Whenever we're dealing with IDs and their references, the best choice you have is _Search & Replace_. This should only be the last resort.
{'fraction_non_alphanumeric': 0.09898565932144106, 'fraction_numerical': 0.023201585636003264, 'mean_word_length': 4.078744819419775, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 14, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '659367', 'n_tokens_mistral': 2818, 'n_tokens_neox': 2555, 'n_words': 1053}