<commit_before># Writing Build Scripts

One of the most common actions that Buildkite steps perform is running shell scripts. These scripts are checked in alongside your code and `pipeline.yml` file. The [Buildkite Agent](/docs/agent) will run your scripts, capture and report the log output, and use the exit status to mark each job, as well as the overall build, as passed or failed.

<%= toc %>

## Configuring Bash

The shell that runs your scripts in Buildkite is a clean Bash prompt with no settings. If you rely on anything from your `~/.bash_profile` or `~/.bashrc` files when you run scripts locally, you'll need to explicitly add the relevant items to your build scripts.

When writing Bash shell scripts there are a number of options you can set to help prevent unexpected errors:

<table>
  <tbody>
  <tr>
    <th><code>e</code></th>
    <td>Exit script immediately if any command returns a non-zero exit status.</td>
  </tr>
  <tr>
    <th><code>u</code></th>
    <td>Exit script immediately if an undefined variable is used (e.g. <code>echo "$UNDEFINED_ENV_VAR"</code>).</td>
  </tr>
  <tr>
    <th><code>o pipefail</code></th>
    <td>Ensure Bash pipelines (e.g. <code>cmd | othercmd</code>) return a non-zero status if any of the commands fail, rather than returning the exit status of the last command in the pipeline.</td>
  </tr>
  <tr>
    <th><code>x</code></th>
    <td>Expand and print each command before executing. See <a href="/docs/builds/writing-build-scripts#debugging-your-environment">Debugging your environment</a> for more information. </td>
  </tr>
  </tbody>
</table>

Bash’s built-in `set` command can be used to enable and disable options. To enable an option use the `-` argument, and to disable an option use the `+` argument.

For example, the following script enables the most commonly used options for build scripts:

```bash
#!/bin/bash

set -euo pipefail

run_tests
```

For a full list of options, see the [Bash Reference Manual](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html).

<section class="Docs__troubleshooting-note">
  <h1>Unbound variable errors</h1>
  <p>Note that while the <code>u</code> option is generally a good default to use for all build scripts, it can cause some tools like <a href="https://rvm.io">rvm</a> to fail with “unbound variable” errors. If you encounter errors, remove the <code>u</code> from the set command in your script.</p>
</section>

## Capturing exit status

Build scripts can sometimes contain commands that shouldn't affect the overall exit status. For example, take the following script:

```bash
#!/bin/bash

run_tests

clean_up
``` 

Running this script will exit with the status returned by the final command, `clean_up`. However, what we really care about is the exit status of the first command, `run_tests`.

By using a variable to store the exit status of `run_tests`, we can run additional commands while still returning the original exit status. For example:

```bash
#!/bin/bash

# Note the lack of 'e' option here
set -uo pipefail

# Run the command we're most interested in
run_tests

# Capture the last exit status in a variable
TEST_STATUS=$?

# Run additional commands
clean_up

# Finish with the status of the original command
exit $TEST_STATUS
```

Using this technique gives you complete control over the exit code of your script, and the final success or failure of your build job.

## Debugging your environment

The first step in debugging your build script is to view the environment variables from the Buildkite web interface:

<%= image "viewing-job-environment-variables.png", width: 1044/2, height: 542/2, alt: 'Screenshot of viewing a job’s environment tab' %>

There may be additional environment variables available in your build job that don’t appear in this list, such as ones set by your own [agent hooks](/docs/agent/hooks). To debug these, you can print them using `echo $SOME_VAR` before the command you're wanting to run. For example:<commit_msg>Clarify why we exclude 'e'<commit_after># Writing Build Scripts

One of the most common actions that Buildkite steps perform is running shell scripts. These scripts are checked in alongside your code and `pipeline.yml` file. The [Buildkite Agent](/docs/agent) will run your scripts, capture and report the log output, and use the exit status to mark each job, as well as the overall build, as passed or failed.

<%= toc %>

## Configuring Bash

The shell that runs your scripts in Buildkite is a clean Bash prompt with no settings. If you rely on anything from your `~/.bash_profile` or `~/.bashrc` files when you run scripts locally, you'll need to explicitly add the relevant items to your build scripts.

When writing Bash shell scripts there are a number of options you can set to help prevent unexpected errors:

<table>
  <tbody>
  <tr>
    <th><code>e</code></th>
    <td>Exit script immediately if any command returns a non-zero exit status.</td>
  </tr>
  <tr>
    <th><code>u</code></th>
    <td>Exit script immediately if an undefined variable is used (e.g. <code>echo "$UNDEFINED_ENV_VAR"</code>).</td>
  </tr>
  <tr>
    <th><code>o pipefail</code></th>
    <td>Ensure Bash pipelines (e.g. <code>cmd | othercmd</code>) return a non-zero status if any of the commands fail, rather than returning the exit status of the last command in the pipeline.</td>
  </tr>
  <tr>
    <th><code>x</code></th>
    <td>Expand and print each command before executing. See <a href="/docs/builds/writing-build-scripts#debugging-your-environment">Debugging your environment</a> for more information. </td>
  </tr>
  </tbody>
</table>

Bash’s built-in `set` command can be used to enable and disable options. To enable an option use the `-` argument, and to disable an option use the `+` argument.

For example, the following script enables the most commonly used options for build scripts:

```bash
#!/bin/bash

set -euo pipefail

run_tests
```

For a full list of options, see the [Bash Reference Manual](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html).

<section class="Docs__troubleshooting-note">
  <h1>Unbound variable errors</h1>
  <p>Note that while the <code>u</code> option is generally a good default to use for all build scripts, it can cause some tools like <a href="https://rvm.io">rvm</a> to fail with “unbound variable” errors. If you encounter errors, remove the <code>u</code> from the set command in your script.</p>
</section>

## Capturing exit status

Build scripts can sometimes contain commands that shouldn't affect the overall exit status. For example, take the following script:

```bash
#!/bin/bash

run_tests

clean_up
``` 

Running this script will exit with the status returned by the final command, `clean_up`. However, what we really care about is the exit status of the first command, `run_tests`.

By using a variable to store the exit status of `run_tests`, we can run additional commands while still returning the original exit status. For example:

```bash
#!/bin/bash

# Note that we don't enable the 'e' option, which would cause the script to 
# immediately exit if 'run_tests' failed.
set -uo pipefail

# Run the command we're most interested in
run_tests

# Capture the last exit status in a variable
TEST_STATUS=$?

# Run additional commands
clean_up

# Finish with the status of the original command
exit $TEST_STATUS
```

Using this technique gives you complete control over the exit code of your script, and the final success or failure of your build job.

## Debugging your environment

The first step in debugging your build script is to view the environment variables from the Buildkite web interface:

<%= image "viewing-job-environment-variables.png", width: 1044/2, height: 542/2, alt: 'Screenshot of viewing a job’s environment tab' %>

There may be additional environment variables available in your build job that don’t appear in this list, such as ones set by your own [agent hooks](/docs/agent/hooks). To debug these, you can print them using `echo $SOME_VAR` before the command you're wanting to run. For example:
{'fraction_non_alphanumeric': 0.09179759990102684, 'fraction_numerical': 0.0027217617221328714, 'mean_word_length': 4.263020833333333, 'pattern_counts': {'":': 0, '<': 115, '<?xml version=': 0, '>': 115, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1776156', 'n_tokens_mistral': 2356, 'n_tokens_neox': 2260, 'n_words': 1113}