<commit_before># Upgrading to Buildkite Agent v3

The Buildkite Agent has changed a lot in v3, but we've tried to keep it as backwards compatible as possible. We'll cover what’s changed and how to upgrade to new agents.

<%= toc %>

## What's changed

* [Plugins](/docs/agents/plugins) for easily sharing functionality between pipelines and customizing how agents behave
* The bootstrap (run as a sub-process for every job) has moved from a [shell script](https://github.com/buildkite/agent/blob/2-6-stable/templates/bootstrap.sh) to a sub-command of the `buildkite-agent bootstrap`. This means it's written in golang and cross-platform.
* Agent meta-data has been renamed to "tags"
* Built-in [Docker and Docker Compose support](/docs/builds/docker-containerized-builds) has been deprecated and moved to dedicated plugins, [docker-compose](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin) and [docker](https://github.com/buildkite-plugins/docker-buildkite-plugin).
* Variable interpolation in pipeline.yml
* Much better Windows support, including .BAT hooks support
* Build annotations
* Default flags for `git clean` include `-x`

## Bootstrap customizations

If you customized your `bootstrap.sh` file, you will need to move the changes to [hooks documentation](hooks), or update your bootstrap.sh to call `buildkite-agent bootstrap`.

## Docker and Docker Compose support

In v2 we supported a variety of environment variables like `BUILDKITE_DOCKER_COMPOSE_CONTAINER` and `BUILDKITE_DOCKER`. These are deprecated in favour of the [docker-compose](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin) and [docker](https://github.com/buildkite-plugins/docker-buildkite-plugin) pipeline plugin.

You can keep using the old environment variables in v3, but they will be removed in v4.

### Steps using `BUILDKITE_DOCKER_COMPOSE_CONTAINER`

This is a step that uses the v2 `BUILDKITE_DOCKER_COMPOSE_CONTAINER` environment variable to run the command in a docker-compose container:

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    env:
      BUILDKITE_DOCKER_COMPOSE_CONTAINER: app
```

To convert it to the [docker-compose plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin)
you would do this:

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    plugins:
      docker-compose#v1.7.0:
        run: app
```

Note that at present these plugins are implemented in bash, so will only work on non-windows platforms. Windows users should continue to use the environment vars.

### Steps using `BUILDKITE_DOCKER`

This is a step that uses the v2 `BUILDKITE_DOCKER` environment variable to run the command in docker container:

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    env:
      BUILDKITE_DOCKER: true
```

There isn't a direct conversion for this at present, you can either add a docker-compose file and use the [docker-compose plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin), or if you want to run your build in a docker container without providing a `Dockerfile` or a `docker-compose` file, you can use the [docker plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin):

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    plugins:
      docker#v1.0.0:
        image: "node:7"
        workdir: /app
```

### Variables in your pipeline.yml

Previously we didn't support variable interpolation like `${MY_VARIABLE_NAME}` or `$MY_VARIABLE_NAME`, so if you have any of these in your previous pipeline's and expect the literal text to render, you will need to escape the dollar signs like `$$MY_VARIABLE_NAME`.

### Default flags for `git clean` include `-x`

Older agents didn't remove files from your working directory that were ignored by git. The new default values for git clean are `-fxdq`. If you've previously overridden your `git-clean-flags` in your config, it might be a good chance to comment them out and use the standard behaviour.

## Upgrading a 2.0 agent

To upgrade you simply install the a new 3.0 agent using the [standard installation methods](/docs/agent/v2/installation). You can use agent tags to create a new queue for 3.0 builds so that you can test it in parallel to your existing agents.

To make installation easy we’ve created packages for each of the major operating systems.

## Upgrading a 3.0 beta agent<commit_msg>Make these top-level sections, they're not Docker related<commit_after># Upgrading to Buildkite Agent v3

The Buildkite Agent has changed a lot in v3, but we've tried to keep it as backwards compatible as possible. We'll cover what’s changed and how to upgrade to new agents.

<%= toc %>

## What's changed

* [Plugins](/docs/agents/plugins) for easily sharing functionality between pipelines and customizing how agents behave
* The bootstrap (run as a sub-process for every job) has moved from a [shell script](https://github.com/buildkite/agent/blob/2-6-stable/templates/bootstrap.sh) to a sub-command of the `buildkite-agent bootstrap`. This means it's written in golang and cross-platform.
* Agent meta-data has been renamed to "tags"
* Built-in [Docker and Docker Compose support](/docs/builds/docker-containerized-builds) has been deprecated and moved to dedicated plugins, [docker-compose](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin) and [docker](https://github.com/buildkite-plugins/docker-buildkite-plugin).
* Variable interpolation in pipeline.yml
* Much better Windows support, including .BAT hooks support
* Build annotations
* Default flags for `git clean` include `-x`

## Bootstrap customizations

If you customized your `bootstrap.sh` file, you will need to move the changes to [hooks documentation](hooks), or update your bootstrap.sh to call `buildkite-agent bootstrap`.

## Docker and Docker Compose support

In v2 we supported a variety of environment variables like `BUILDKITE_DOCKER_COMPOSE_CONTAINER` and `BUILDKITE_DOCKER`. These are deprecated in favour of the [docker-compose](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin) and [docker](https://github.com/buildkite-plugins/docker-buildkite-plugin) pipeline plugin.

You can keep using the old environment variables in v3, but they will be removed in v4.

### Steps using `BUILDKITE_DOCKER_COMPOSE_CONTAINER`

This is a step that uses the v2 `BUILDKITE_DOCKER_COMPOSE_CONTAINER` environment variable to run the command in a docker-compose container:

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    env:
      BUILDKITE_DOCKER_COMPOSE_CONTAINER: app
```

To convert it to the [docker-compose plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin)
you would do this:

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    plugins:
      docker-compose#v1.7.0:
        run: app
```

Note that at present these plugins are implemented in bash, so will only work on non-windows platforms. Windows users should continue to use the environment vars.

### Steps using `BUILDKITE_DOCKER`

This is a step that uses the v2 `BUILDKITE_DOCKER` environment variable to run the command in docker container:

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    env:
      BUILDKITE_DOCKER: true
```

There isn't a direct conversion for this at present, you can either add a docker-compose file and use the [docker-compose plugin](https://github.com/buildkite-plugins/docker-compose-buildkite-plugin), or if you want to run your build in a docker container without providing a `Dockerfile` or a `docker-compose` file, you can use the [docker plugin](https://github.com/buildkite-plugins/docker-buildkite-plugin):

```yaml
steps:
  - label: ':hammer: Tests'
    command: 'scripts/tests.sh'
    plugins:
      docker#v1.0.0:
        image: "node:7"
        workdir: /app
```

## Variables in your pipeline.yml

Previously we didn't support variable interpolation like `${MY_VARIABLE_NAME}` or `$MY_VARIABLE_NAME`, so if you have any of these in your previous pipeline's and expect the literal text to render, you will need to escape the dollar signs like `$$MY_VARIABLE_NAME`.

## Default flags for `git clean` include `-x`

Older agents didn't remove files from your working directory that were ignored by git. The new default values for git clean are `-fxdq`. If you've previously overridden your `git-clean-flags` in your config, it might be a good chance to comment them out and use the standard behaviour.

## Upgrading a 2.0 agent

To upgrade you simply install the a new 3.0 agent using the [standard installation methods](/docs/agent/v2/installation). You can use agent tags to create a new queue for 3.0 builds so that you can test it in parallel to your existing agents.

To make installation easy we’ve created packages for each of the major operating systems.

## Upgrading a 3.0 beta agent
{'fraction_non_alphanumeric': 0.08867378727191812, 'fraction_numerical': 0.0055629728526924785, 'mean_word_length': 4.44457904300424, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 16, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6255034', 'n_tokens_mistral': 2734, 'n_tokens_neox': 2563, 'n_words': 1066}