<commit_before># Docker Hub

[Docker Hub](https://hub.docker.com/) is a public registry of docker images,
hosting popular images used in many build pipelines.

On 2nd November 2020, Docker Hub introduced [strict rate
limits](https://docs.docker.com/docker-hub/download-rate-limit/) on image
downloads by unauthenticated clients, and authenticated clients on a free plan.
For Buildkite customers using images hosted on Docker Hub, this results in intermittent job failure.

How to prevent job failure caused by the Docker Hub rates limits depends on exactly how you are using Docker images, here are a few solutions for common scenarios.

{:toc}

## Elastic CI Stack for AWS, authenticating with a paid Docker Hub account

If you're using the [Elastic CI Stack for AWS](https://github.com/buildkite/elastic-ci-stack-for-aws), you can authenticate with Docker Hub by [adding two
environment variables](https://github.com/buildkite/elastic-ci-stack-for-aws#docker-registry-support) to your secrets
bucket and accessing them from your build.

Add your Docker Hub credentials to one of the following two environment hooks, which are downloaded at the start of each job:

* `/env` - An agent environment hook, run for every job the agent runs
* `/{pipeline-slug}/env` - An agent environment hook, specific to a pipeline

Either one of these could be configured with Docker Hub credentials to ensure
Docker Hub requests are authenticated:

```bash
#!/bin/bash

DOCKER_LOGIN_USER="the-user-name"
DOCKER_LOGIN_PASSWORD="<PASSWORD>"
```

## Other Buildkite agents authenticating with a paid Docker Hub account

All agents check the local filesystem for [hook scripts to execute during a job](/docs/agent/v3/hooks).

A [pre-command hook](https://buildkite.com/docs/agent/v3/hooks#available-hooks) script like this is one option for authenticating with Docker Hub, and can be configured to fetch credentials from the system you use to store them in:

```bash
#!/bin/bash

echo "~~~ Logging into Docker Hub"
docker login --username "the-user-name" --password-stdin << "<PASSWORD>"
```

## Using docker-login plugin to authenticate with a paid Docker Hub account

The [docker-login plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin/) can perform the authentication in just the jobs that need it.

Start by setting the password in an [agent environment hook](/docs/agent/v3/hooks):

    #!/bin/bash

    DOCKER_HUB_PASSWORD="<PASSWORD>"

Then add the plugin to pipeline YAML steps that need it:

```yaml
    steps:
      - command: ./run_build.sh
        plugins:
          - docker-login#v2.0.1:
              username: the-user-name
              password-env: DOCKER_HUB_PASSWORD
```

## Mirroring Docker images into Google Container Registry

If you're using Google Cloud, Google has some documentation on [mirroring
Docker Hub images](https://cloud.google.com/container-registry/docs/migrate-external-containers) to a private Google Container Registry (GCR).

This approach requires:

1. A regular process (e.g. nightly) that mirrors the Docker images you need
2. Updating all pipelines to use the mirrored GCR image (e.g. from `nginx:1.14.2`
   to `gcr.io/<GCR_PROJECT>/nginx:1.14.2`)

## Mirroring Docker images into AWS Elastic Container Registry

AWS doesn't have specific documentation, however a similar solution to the one proposed by Google Cloud Platform (GCP)
above would work with AWS Elastic Container Registry (ECR) as well:

1. A regular process (e.g. nightly) that mirrors the Docker images you need
2. Updating all pipelines to use the mirrored ECR image instead of the original one

## Configuring the docker daemon to use the GCR mirror of popular Docker Hub images

GCP host a mirror of popular Docker Hub images in the `mirror.gcr.io` registry.

For agents running on GCP, it's possible to configure docker to try the mirror
first, and transparently fall back to the public Docker Hub registry when the
mirror doesn't have an image. Google have [documented how to set this
us](https://cloud.google.com/container-registry/docs/pulling-cached-images).

This will avoid the rate limits for many, but Google don't
guarantee which images will be on the mirror, so depending on the specific
images in use you may continue to hit Docker Hub rate limits.

## Running a read-through caching registry

There are two popular options for running a private caching docker registry,
where requests for missing images result in the image being fetched from an
origin registry (like Docker Hub).<commit_msg>a few very small copy edits<commit_after># Docker Hub

[Docker Hub](https://hub.docker.com/) is a public registry of docker images,
hosting popular images used in many build pipelines.

On 2nd November 2020, Docker Hub introduced [strict rate
limits](https://docs.docker.com/docker-hub/download-rate-limit/) on image
downloads by unauthenticated clients, and authenticated clients on a free plan.
For Buildkite customers using images hosted on Docker Hub, this results in intermittent job failures.

How to prevent job failure caused by the Docker Hub rates limits depends on exactly how you are using Docker images, here are a few solutions for common scenarios.

{:toc}

## Elastic CI Stack for AWS, authenticating with a paid Docker Hub account

If you're using the [Elastic CI Stack for AWS](https://github.com/buildkite/elastic-ci-stack-for-aws), you can authenticate with Docker Hub by [adding two
environment variables](https://github.com/buildkite/elastic-ci-stack-for-aws#docker-registry-support) to your secrets
bucket and accessing them from your build.

Add your Docker Hub credentials to one of the following two environment hooks, which are downloaded at the start of each job:

* `/env` - An agent environment hook, run for every job the agent runs
* `/{pipeline-slug}/env` - An agent environment hook, specific to a pipeline

Either one of these could be configured with Docker Hub credentials to ensure
Docker Hub requests are authenticated:

```bash
#!/bin/bash

DOCKER_LOGIN_USER="the-user-name"
DOCKER_LOGIN_PASSWORD="<PASSWORD>"
```

## Other Buildkite agents authenticating with a paid Docker Hub account

All agents check the local filesystem for [hook scripts to execute during a job](/docs/agent/v3/hooks).

A [pre-command hook](https://buildkite.com/docs/agent/v3/hooks#available-hooks) script like this is one option for authenticating with Docker Hub, and can be configured to fetch credentials from the system you use to store them in:

```bash
#!/bin/bash

echo "~~~ Logging into Docker Hub"
docker login --username "the-user-name" --password-stdin << "the-password"
```

## Using docker-login plugin to authenticate with a paid Docker Hub account

The [docker-login plugin](https://github.com/buildkite-plugins/docker-login-buildkite-plugin/) can perform the authentication in just the steps that need it.

Start by setting the password in an [agent environment hook](/docs/agent/v3/hooks):

    #!/bin/bash

    DOCKER_HUB_PASSWORD="<PASSWORD>"

Then add the plugin to pipeline YAML steps that need it:

```yaml
    steps:
      - command: ./run_build.sh
        plugins:
          - docker-login#v2.0.1:
              username: the-user-name
              password-env: DOCKER_HUB_PASSWORD
```

## Mirroring Docker images into Google Container Registry

If you're using Google Cloud, Google has some documentation on [mirroring
Docker Hub images](https://cloud.google.com/container-registry/docs/migrate-external-containers) to a private Google Container Registry (GCR).

This approach requires:

1. A regular process (e.g. nightly) that mirrors the Docker images you need
2. Updating all pipelines to use the mirrored GCR image (e.g. from `nginx:1.14.2`
   to `gcr.io/<GCR_PROJECT>/nginx:1.14.2`)

## Mirroring Docker images into AWS Elastic Container Registry

AWS doesn't have specific documentation, however a similar solution to the one proposed by Google Cloud Platform (GCP)
above would work with AWS Elastic Container Registry (ECR) as well:

1. A regular process (e.g. nightly) that mirrors the Docker images you need
2. Updating all pipelines to use the mirrored ECR image instead of the original one

## Configuring the docker daemon to use the GCR mirror of popular Docker Hub images

GCP host a mirror of popular Docker Hub images in the `mirror.gcr.io` registry.

For agents running on GCP, it's possible to configure docker to try the mirror
first, and transparently fall back to the public Docker Hub registry when the
mirror doesn't have an image. Google have [documented how to set this
up](https://cloud.google.com/container-registry/docs/pulling-cached-images).

This will avoid the rate limits for many, but Google don't
guarantee which images will be on the mirror, so depending on the specific
images in use you may continue to hit Docker Hub rate limits.

## Running a read-through caching registry

There are two popular options for running a private caching docker registry,
where requests for missing images result in the image being fetched from an
origin registry (like Docker Hub).
{'fraction_non_alphanumeric': 0.07486218302094819, 'fraction_numerical': 0.005071664829106946, 'mean_word_length': 4.307782328847279, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 10, 'https://': 16, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5546972', 'n_tokens_mistral': 2596, 'n_tokens_neox': 2419, 'n_words': 1168}