<commit_before># Deploying Prow

This document will walk you through deploying your own Prow instance to a new Kubernetes cluster. If you encounter difficulties, please open an issue so that we can make this process easier.

Prow runs in any kubernetes cluster. Our `tackle` utility helps deploy it correctly, or you can perform each of the steps manually.

Both of these are focused on [Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) but should work on any kubernetes distro with no/minimal changes.

## GitHub bot account

Before using `tackle` or deploying prow manually, ensure you have created a
GitHub account for prow to use.  Prow will ignore most GitHub events generated
by this account, so it is important this account be separate from any users or
automation you wish to interact with prow. For example, you still need to do
this even if you'd just setting up a prow instance to work against your own
personal repos.

1. Ensure the bot user has the following permissions
    - Write access to the repos you plan on handling
    - Owner access (and org membership) for the orgs you plan on handling (note
      it is possible to handle specific repos in an org without this)
1. Create a [personal access token][1] for the GitHub bot account, adding the
   following scopes (more details [here][8])
    - Must have the `public_repo` and `repo:status` scopes
    - Add the `repo` scope if you plan on handing private repos
    - Add the `admin:org_hook` scope if you plan on handling a github org
1. Set this token aside for later (we'll assume you wrote it to a file on your
   workstation at `/path/to/github/token`)

## Tackle deployment

Prow's `tackle` utility walks you through deploying a new instance of prow in a couple minutes, try it out!

You need a few things:

1. [`bazel`](https://bazel.build/) build tool installed and working
1. The prow `tackle` utility. It is recommended to use it by running `bazel run //prow/cmd/tackle` from `test-infra` directory, alternatively you can install it by running `go get -u k8s.io/test-infra/prow/cmd/tackle` (in that case you would also need go installed and working).
**Note**: Creating the `tackle` utility assumes you have the `gcloud` application in your `$PATH`,
if you are doing this on another cloud skip to the **Manual deployment** below.
1. Optionally, credentials to a Kubernetes cluster (otherwise, `tackle` will help you create on GCP)

To install prow run the following from the `test-infra` directory and follow the on-screen instructions:

```sh
# Ideally use https://bazel.build, alternatively try:
#   go get -u k8s.io/test-infra/prow/cmd/tackle && tackle
$ bazel run //prow/cmd/tackle
```

This will help you through the following steps:

* Choosing a kubectl context (and creating a cluster / getting its credentials if necessary)
* Deploying prow into that cluster
* Configuring GitHub to send prow webhooks for your repos. This is where you'll provide the absolute `/path/to/github/token`

See the [Next Steps](#next-steps) section after running this utility.

## Manual deployment

If you do not want to use the `tackle` utility above, here are the manual set of commands tackle will run.

Prow runs in a kubernetes cluster, so first figure out which cluster you want to deploy prow into.
If you already have a cluster created you can skip to the **Create cluster role bindings** step.

### Create the cluster

You can use the [GCP cloud console](https://console.cloud.google.com/) to set up a project and [create a new Kubernetes Engine cluster](https://console.cloud.google.com/kubernetes).

I'm assuming that `PROJECT` and `ZONE` environment variables are set, if you are using
GCP. Skip this step if you are using another service to host your Kubernetes cluster.

```sh
$ export PROJECT=your-project
$ export ZONE=us-west1-a
```

Run the following to create the cluster. This will also set up `kubectl` to
point to the new cluster on GCP.

```sh
$ gcloud container --project "${PROJECT}" clusters create prow \
  --zone "${ZONE}" --machine-type n1-standard-4 --num-nodes 2
```

### Create cluster role bindings

As of 1.8 Kubernetes uses [Role-Based Access Control (“RBAC”)](https://kubernetes.io/docs/admin/authorization/rbac/) to drive authorization decisions, allowing `cluster-admin` to dynamically configure policies.
To create cluster resources you need to grant a user `cluster-admin` role in all namespaces for the cluster.

For Prow on GCP, you can use the following command.

```sh
$ kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin --user $(gcloud config get-value account)
```

For Prow on other platforms, the following command will likely work.

```sh
$ kubectl create clusterrolebinding cluster-admin-binding-"${USER}" \
  --clusterrole=cluster-admin --user="${USER}"
```

On some platforms the `USER` variable may not map correctly to the user
in-cluster. If you see an error of the following form, this is likely the case.

```sh
Error from server (Forbidden): error when creating
"config/prow/cluster/starter-gcs.yaml": roles.rbac.authorization.k8s.io "<account>" is
forbidden: attempt to grant extra privileges:
[PolicyRule{Resources:["pods/log"], APIGroups:[""], Verbs:["get"]}
PolicyRule{Resources:["prowjobs"], APIGroups:["prow.k8s.io"], Verbs:["get"]}
APIGroups:["prow.k8s.io"], Verbs:["list"]}] user=&{<CLUSTER_USER>
[system:authenticated] map[]}...
```

Run the previous command substituting `USER` with `CLUSTER_USER` from the error
message above to solve this issue.

```sh
$ kubectl create clusterrolebinding cluster-admin-binding-"<CLUSTER_USER>" \
  --clusterrole=cluster-admin --user="<CLUSTER_USER>"
```

There are [relevant docs on Kubernetes Authentication](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies) that may help if neither of the above work.

### Create the GitHub secrets

You will need two secrets to talk to GitHub. The `hmac-token` is the token that
you give to GitHub for validating webhooks. Generate it using any reasonable
randomness-generator, eg `openssl rand -hex 20`.

```sh
$ openssl rand -hex 20 > /path/to/hook/secret
$ kubectl create secret -n prow generic hmac-token --from-file=hmac=/path/to/hook/secret
```

The `github-token` is the *Personal access token* you created above for the [GitHub bot account].
If you need to create one, go to <https://github.com/settings/tokens>.

```sh
kubectl create secret -n prow generic github-token --from-file=token=/path/to/github/token
```

### Update the sample manifest

There are two sample manifests to get you started:
* [`starter-s3.yaml`](/config/prow/cluster/starter-s3.yaml) sets up a minio as blob storage for logs and is particularly well suited to quickly get something working
* [`starter-gcs.yaml`](/config/prow/cluster/starter-gcs.yaml) uses GCS as blob storage and requires additional configuration to set up the bucket and ServiceAccounts. See [this](#configure-a-gcs-bucket) for details.

**Note**: It will deploy prow in the `prow` namespace of the cluster.

Regardless of which object storage you choose, the below adjustments are always needed:

* The github token by replacing the `<<insert-token-here>>` string
* The hmac token by replacing the `<< insert-hmac-token-here >>` string
* The domain by replacing the `<< your-domain.com >>` string
* Optionally, you can update the `cert-manager.io/cluster-issuer:` annotation if you use cert-manager
* Your github organization(s) by replacing the `<< your_github_org >>` string

### Add the prow components to the cluster

Apply the manifest you edited above by executing one of the following two commands:


* `kubectl apply -f config/prow/cluster/starter-s3.yaml`
* `kubectl apply -f config/prow/cluster/starter-gcs.yaml`<commit_msg>Update getting_started_deploy with info on needed PersistentVolumes<commit_after># Deploying Prow

This document will walk you through deploying your own Prow instance to a new Kubernetes cluster. If you encounter difficulties, please open an issue so that we can make this process easier.

Prow runs in any kubernetes cluster. Our `tackle` utility helps deploy it correctly, or you can perform each of the steps manually.

Both of these are focused on [Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) but should work on any kubernetes distro with no/minimal changes.

## GitHub bot account

Before using `tackle` or deploying prow manually, ensure you have created a
GitHub account for prow to use.  Prow will ignore most GitHub events generated
by this account, so it is important this account be separate from any users or
automation you wish to interact with prow. For example, you still need to do
this even if you'd just setting up a prow instance to work against your own
personal repos.

1. Ensure the bot user has the following permissions
    - Write access to the repos you plan on handling
    - Owner access (and org membership) for the orgs you plan on handling (note
      it is possible to handle specific repos in an org without this)
1. Create a [personal access token][1] for the GitHub bot account, adding the
   following scopes (more details [here][8])
    - Must have the `public_repo` and `repo:status` scopes
    - Add the `repo` scope if you plan on handing private repos
    - Add the `admin:org_hook` scope if you plan on handling a github org
1. Set this token aside for later (we'll assume you wrote it to a file on your
   workstation at `/path/to/github/token`)

## Tackle deployment

Prow's `tackle` utility walks you through deploying a new instance of prow in a couple minutes, try it out!

You need a few things:

1. [`bazel`](https://bazel.build/) build tool installed and working
1. The prow `tackle` utility. It is recommended to use it by running `bazel run //prow/cmd/tackle` from `test-infra` directory, alternatively you can install it by running `go get -u k8s.io/test-infra/prow/cmd/tackle` (in that case you would also need go installed and working).
**Note**: Creating the `tackle` utility assumes you have the `gcloud` application in your `$PATH`,
if you are doing this on another cloud skip to the **Manual deployment** below.
1. Optionally, credentials to a Kubernetes cluster (otherwise, `tackle` will help you create on GCP)

To install prow run the following from the `test-infra` directory and follow the on-screen instructions:

```sh
# Ideally use https://bazel.build, alternatively try:
#   go get -u k8s.io/test-infra/prow/cmd/tackle && tackle
$ bazel run //prow/cmd/tackle
```

This will help you through the following steps:

* Choosing a kubectl context (and creating a cluster / getting its credentials if necessary)
* Deploying prow into that cluster
* Configuring GitHub to send prow webhooks for your repos. This is where you'll provide the absolute `/path/to/github/token`

See the [Next Steps](#next-steps) section after running this utility.

## Manual deployment

If you do not want to use the `tackle` utility above, here are the manual set of commands tackle will run.

Prow runs in a kubernetes cluster, so first figure out which cluster you want to deploy prow into.
If you already have a cluster created you can skip to the **Create cluster role bindings** step.

### Create the cluster

You can use the [GCP cloud console](https://console.cloud.google.com/) to set up a project and [create a new Kubernetes Engine cluster](https://console.cloud.google.com/kubernetes).

I'm assuming that `PROJECT` and `ZONE` environment variables are set, if you are using
GCP. Skip this step if you are using another service to host your Kubernetes cluster.

```sh
$ export PROJECT=your-project
$ export ZONE=us-west1-a
```

Run the following to create the cluster. This will also set up `kubectl` to
point to the new cluster on GCP.

```sh
$ gcloud container --project "${PROJECT}" clusters create prow \
  --zone "${ZONE}" --machine-type n1-standard-4 --num-nodes 2
```

### Create cluster role bindings

As of 1.8 Kubernetes uses [Role-Based Access Control (“RBAC”)](https://kubernetes.io/docs/admin/authorization/rbac/) to drive authorization decisions, allowing `cluster-admin` to dynamically configure policies.
To create cluster resources you need to grant a user `cluster-admin` role in all namespaces for the cluster.

For Prow on GCP, you can use the following command.

```sh
$ kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin --user $(gcloud config get-value account)
```

For Prow on other platforms, the following command will likely work.

```sh
$ kubectl create clusterrolebinding cluster-admin-binding-"${USER}" \
  --clusterrole=cluster-admin --user="${USER}"
```

On some platforms the `USER` variable may not map correctly to the user
in-cluster. If you see an error of the following form, this is likely the case.

```sh
Error from server (Forbidden): error when creating
"config/prow/cluster/starter-gcs.yaml": roles.rbac.authorization.k8s.io "<account>" is
forbidden: attempt to grant extra privileges:
[PolicyRule{Resources:["pods/log"], APIGroups:[""], Verbs:["get"]}
PolicyRule{Resources:["prowjobs"], APIGroups:["prow.k8s.io"], Verbs:["get"]}
APIGroups:["prow.k8s.io"], Verbs:["list"]}] user=&{<CLUSTER_USER>
[system:authenticated] map[]}...
```

Run the previous command substituting `USER` with `CLUSTER_USER` from the error
message above to solve this issue.

```sh
$ kubectl create clusterrolebinding cluster-admin-binding-"<CLUSTER_USER>" \
  --clusterrole=cluster-admin --user="<CLUSTER_USER>"
```

There are [relevant docs on Kubernetes Authentication](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies) that may help if neither of the above work.

### Create the GitHub secrets

You will need two secrets to talk to GitHub. The `hmac-token` is the token that
you give to GitHub for validating webhooks. Generate it using any reasonable
randomness-generator, eg `openssl rand -hex 20`.

```sh
$ openssl rand -hex 20 > /path/to/hook/secret
$ kubectl create secret -n prow generic hmac-token --from-file=hmac=/path/to/hook/secret
```

The `github-token` is the *Personal access token* you created above for the [GitHub bot account].
If you need to create one, go to <https://github.com/settings/tokens>.

```sh
kubectl create secret -n prow generic github-token --from-file=token=/path/to/github/token
```

### Update the sample manifest

There are two sample manifests to get you started:
* [`starter-s3.yaml`](/config/prow/cluster/starter-s3.yaml) sets up a minio as blob storage for logs and is particularly well suited to quickly get something working. NOTE: this method requires 2 PVs of 100Gi each.
* [`starter-gcs.yaml`](/config/prow/cluster/starter-gcs.yaml) uses GCS as blob storage and requires additional configuration to set up the bucket and ServiceAccounts. See [this](#configure-a-gcs-bucket) for details.

**Note**: It will deploy prow in the `prow` namespace of the cluster.

Regardless of which object storage you choose, the below adjustments are always needed:

* The github token by replacing the `<<insert-token-here>>` string
* The hmac token by replacing the `<< insert-hmac-token-here >>` string
* The domain by replacing the `<< your-domain.com >>` string
* Optionally, you can update the `cert-manager.io/cluster-issuer:` annotation if you use cert-manager
* Your github organization(s) by replacing the `<< your_github_org >>` string

### Add the prow components to the cluster

Apply the manifest you edited above by executing one of the following two commands:


* `kubectl apply -f config/prow/cluster/starter-s3.yaml`
* `kubectl apply -f config/prow/cluster/starter-gcs.yaml`
{'fraction_non_alphanumeric': 0.09330615884338578, 'fraction_numerical': 0.0035666518056174765, 'mean_word_length': 4.429460580912863, 'pattern_counts': {'":': 2, '<': 29, '<?xml version=': 0, '>': 31, 'https://': 16, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1530923', 'n_tokens_mistral': 4574, 'n_tokens_neox': 4355, 'n_words': 2052}