<commit_before># Configuration Options

## Global configuration

Hokusai requires IAM configuration for working with AWS resources. It uses the Boto3 library and [loads configuration](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#configuring-credentials) from a local file or via the environment variables `$AWS_ACCESS_KEY_ID` and `$AWS_SECRET_ACCESS_KEY`.

You can set overrides for certain command options and project-specific configuration varaiables in the environment as well.  See below for project-specific configuration variables and their enviroment overrides.  The order of precence for these variables is as follows:

- If the configuration variable is bound to a command-line option, the option supplied on the command line always takes precendence.
- If the configuration variable can be overridden by an environment variable, the environment variable takes precendence if set.

When running `hokusai configure` the following files are created:

* `kubectl` installed on your $PATH
* Your organization-specific `~/.kube/config` kubectl configuration file

## Project configuration

When running `hokusai setup` the following files are created:

* `./hokusai/config.yml` contains project-specific configuration variables.  It is loaded a top-level hash that accepts the following keys:

    - `project-name`: <string> (required) - The project name
    - `hokusai-required-version`: <string> (optional) - A [PEP-440 version specifier string](https://www.python.org/dev/peps/pep-0440/#version-specifiers).  Hokusai will raise an error when running commands if its current version does not satisfy these version specifications.  For example: `~=0.5`, `==0.5.1`, `>=0.4.0,<0.4.6`, `!=0.1.*` are all valid version specifier strings
    - `pre-build`: <string> (optional) - A pre-build hook - useful to inject dynamic environment variables into the build, for example: `export COMMIT_HASH=$(git rev-parse HEAD)`
    - `post-build`: <string> (optional) - A post-build hook - useful for image post-processing
    - `pre-deploy`: <string> (optional) - A pre-deploy hook - useful to enforce migrations
    - `post-deploy`: <string> (optional) - A post-deploy hook - useful for deploy notifications
    - `git-remote`: <string> (optional) - Push deployment tags to git remote when invoking the `hokusai [staging|production|review_app] deploy` or the `hokusai pipeline promote` commands.  Can either be a local alias like 'origin' or a URI like `git@github.com:artsy/hokusai.git`.  Bound to the `--git-remote` option for these commands.
    - `run-tty`: <boolean> (optional) - Attach the terminal to your shell session when invoking `hokusai [staging|production|review_app] run`.  Bound to the `--tty` option for this command, overridden by the `HOKUSAI_RUN_TTY` env var.
    - `follow-logs`: <boolean> (optional) - Follow log output when invoking `hokusai [staging|production|review_app] logs`.  Bound to the `--follow` option for this command, overridden by the `HOKUSAI_FOLLOW_LOGS` env var.
    - `tail-logs`: <integer> (optional) - Tail N lines of log output when invoking `hokusai [staging|production|review_app] logs`.  Bound to the `--tail` option for this command, overridden by the `HOKUSAI_TAIL_LOGS` env var.
    - `run-constraints`: <list of kubernetes label selector strings> - Constrain run containers to Kubernetes nodes matching the label selectors in the form `key=value` by setting the `nodeSelector` field on the container's spec. Bound to the `--constrint` option for `hokusai [staging|production|review_app] run` as well as containers run via the `--migration` flag as well as `pre-deploy` / `post-deploy` hooks triggered by `hokusai [staging|production|review_app] deploy` or `hokusai pipeline promote`.  Overriden by the `HOKUSAI_RUN_CONSTRAINTS` env var.
    - `always-verbose`: <boolean> (optional) - Always pront verbose output.  Bound to the `--verbose` option for various commands, overridden by the `HOKUSAI_ALWAYS_VERBOSE` env var.
<commit_msg>update docs for git-remote config option as it does not apply to review apps as well as run-constraints set via an env var<commit_after># Configuration Options

## Global configuration

Hokusai requires IAM configuration for working with AWS resources. It uses the Boto3 library and [loads configuration](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#configuring-credentials) from a local file or via the environment variables `$AWS_ACCESS_KEY_ID` and `$AWS_SECRET_ACCESS_KEY`.

You can set overrides for certain command options and project-specific configuration varaiables in the environment as well.  See below for project-specific configuration variables and their enviroment overrides.  The order of precence for these variables is as follows:

- If the configuration variable is bound to a command-line option, the option supplied on the command line always takes precendence.
- If the configuration variable can be overridden by an environment variable, the environment variable takes precendence if set.

When running `hokusai configure` the following files are created:

* `kubectl` installed on your $PATH
* Your organization-specific `~/.kube/config` kubectl configuration file

## Project configuration

When running `hokusai setup` the following files are created:

* `./hokusai/config.yml` contains project-specific configuration variables.  It is loaded a top-level hash that accepts the following keys:

    - `project-name`: <string> (required) - The project name
    - `hokusai-required-version`: <string> (optional) - A [PEP-440 version specifier string](https://www.python.org/dev/peps/pep-0440/#version-specifiers).  Hokusai will raise an error when running commands if its current version does not satisfy these version specifications.  For example: `~=0.5`, `==0.5.1`, `>=0.4.0,<0.4.6`, `!=0.1.*` are all valid version specifier strings
    - `pre-build`: <string> (optional) - A pre-build hook - useful to inject dynamic environment variables into the build, for example: `export COMMIT_HASH=$(git rev-parse HEAD)`
    - `post-build`: <string> (optional) - A post-build hook - useful for image post-processing
    - `pre-deploy`: <string> (optional) - A pre-deploy hook - useful to enforce migrations
    - `post-deploy`: <string> (optional) - A post-deploy hook - useful for deploy notifications
    - `git-remote`: <string> (optional) - Push deployment tags to git remote when invoking the `hokusai [staging|production] deploy` or the `hokusai pipeline promote` commands.  Can either be a local alias like 'origin' or a URI like `<EMAIL>:artsy/hokusai.git`.  Bound to the `--git-remote` option for these commands.
    - `run-tty`: <boolean> (optional) - Attach the terminal to your shell session when invoking `hokusai [staging|production|review_app] run`.  Bound to the `--tty` option for this command, overridden by the `HOKUSAI_RUN_TTY` env var.
    - `follow-logs`: <boolean> (optional) - Follow log output when invoking `hokusai [staging|production|review_app] logs`.  Bound to the `--follow` option for this command, overridden by the `HOKUSAI_FOLLOW_LOGS` env var.
    - `tail-logs`: <integer> (optional) - Tail N lines of log output when invoking `hokusai [staging|production|review_app] logs`.  Bound to the `--tail` option for this command, overridden by the `HOKUSAI_TAIL_LOGS` env var.
    - `run-constraints`: <list of kubernetes label selector strings> - Constrain run containers to Kubernetes nodes matching the label selectors in the form `key=value` by setting the `nodeSelector` field on the container's spec. Bound to the `--constrint` option for `hokusai [staging|production|review_app] run` as well as containers run via the `--migration` flag as well as `pre-deploy` / `post-deploy` hooks triggered by `hokusai [staging|production|review_app] deploy` or `hokusai pipeline promote`.  Overriden by the `HOKUSAI_RUN_CONSTRAINTS` env var, in which case a list is parsed from a comma-delimited string.
    - `always-verbose`: <boolean> (optional) - Always pront verbose output.  Bound to the `--verbose` option for various commands, overridden by the `HOKUSAI_ALWAYS_VERBOSE` env var.

{'fraction_non_alphanumeric': 0.09384558913495657, 'fraction_numerical': 0.005628288266242506, 'mean_word_length': 4.621733149931225, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4985008', 'n_tokens_mistral': 2396, 'n_tokens_neox': 2294, 'n_words': 963}