<commit_before># Manifest factorization through `.flux.yaml` configuration files

## Enabling search of `.flux.yaml` files

To enable it supply the command-line flag `--manifest-generation=true` to `fluxd`.

## Goal

It is a common pattern to run very similar resources in separate clusters. There are various scenarios in which this is
required, the two main ones being:

* Having a staging/canary cluster and production cluster. The resources from the staging cluster are regularly promoted
  to the production cluster. In addition, there are long-term differences between the cluster resources (e.g. different
  security keys, different database endpoints etc ...).
* Federation. Different clusters run in separate regions, usually with very similar resources but different
  configurations.

The main goal of `.flux.yaml` configuration files is to help deploying similar resources/clusters:

* with minimal replication of resource definitions
* while keeping Flux neutral about the factorization technology used

## File-access behaviour in Flux

Flux performs two types of actions on raw manifest files from the Git repository:

1. Read manifest files when performing a sync operation (i.e making sure that the status of the cluster reflects what's
   in the manifest files, adjusting it if necessary)
2. Update the manifest files of [workload](fluxctl.html#workloads).
   Specifically, flux can update:
    * container images, when releasing a new image version. A release can happen manually or automatically, when a new
      container image is pushed to a repository.
    * annotations, which establish the release policy of a workload (e.g. whether it should be automatically released,
      whether it should be locked from releasing, what image tags should be considered for automated releases …)

Flux can be configured to confine the scope of (1) and (2):

* To specific (sub)directories (flag `--git-path`)
* To a Git branch other than `master` (flag `--git-branch`)

## Abstracting out file-access: generators and updaters

Flux allows you to declare configuration files to override file operations (1) and (2), by declaring commands which
perform equivalent actions.

The configuration files are formatted in `YAML` and named `.flux.yaml`. They must be located on the Git repository
(more on this later).

A `commandUpdated` `.flux.yaml` file has the following loosely specified format:

```yaml
version: X
commandUpdated:
  generators:
    - command: generator_command1 g1arg1 g1arg2 ...
    - command: generator_command2 g2arg2 g2arg2 ...
  updaters:
    - containerImage:
        command: containerImage_updater_command1 ciu1arg1 ciu1arg2 ...
      policy:
        command: policy_updater_command1 pu1arg1 pu1arg2 ...
    - containerImage:
        command: containerImage_updater_command2 ciu2arg1 ciu2arg2 ...
      policy:
        command: policy_updater_command2 pu2arg1 pu2arg2 ...
```

> **Note:** For a simpler approach to updates, Flux provides a `patchUpdated` configuration file variant.

The file above is versioned (in order to account for future file format changes). Current version is `1`,
which is enforced.

Also, the file contains two generators (declared in the `generators` entry), used to generate manifests and two updaters
(declared in the `updaters` entry), used to update resources in the Git repository.

The generators are meant as an alternative to Flux manifest reads (1). Each updater is split into a `containerImage`
command and a `policy` command, covering the corresponding two types of workload manifest updates mentioned in (2).

> **Note** Update commands operate on policies, rather than annotations. That is for two reasons:
>
> * It is an implementation detail for Kubernetes manifests specifically that policies are represented as annotations.
> * Some configurations (even those for Kubernetes clusters) may encode policies symbolically.

Here is a specific `.flux.yaml` example, declaring a generator and an updater using [Kustomize](https://github.com/kubernetes-sigs/kustomize)
(see [https://github.com/fluxcd/flux-kustomize-example](https://github.com/fluxcd/flux-kustomize-example)<commit_msg>Fix link to flux-kustomize-example<commit_after># Manifest factorization through `.flux.yaml` configuration files

## Enabling search of `.flux.yaml` files

To enable it supply the command-line flag `--manifest-generation=true` to `fluxd`.

## Goal

It is a common pattern to run very similar resources in separate clusters. There are various scenarios in which this is
required, the two main ones being:

* Having a staging/canary cluster and production cluster. The resources from the staging cluster are regularly promoted
  to the production cluster. In addition, there are long-term differences between the cluster resources (e.g. different
  security keys, different database endpoints etc ...).
* Federation. Different clusters run in separate regions, usually with very similar resources but different
  configurations.

The main goal of `.flux.yaml` configuration files is to help deploying similar resources/clusters:

* with minimal replication of resource definitions
* while keeping Flux neutral about the factorization technology used

## File-access behaviour in Flux

Flux performs two types of actions on raw manifest files from the Git repository:

1. Read manifest files when performing a sync operation (i.e making sure that the status of the cluster reflects what's
   in the manifest files, adjusting it if necessary)
2. Update the manifest files of [workload](fluxctl.html#workloads).
   Specifically, flux can update:
    * container images, when releasing a new image version. A release can happen manually or automatically, when a new
      container image is pushed to a repository.
    * annotations, which establish the release policy of a workload (e.g. whether it should be automatically released,
      whether it should be locked from releasing, what image tags should be considered for automated releases …)

Flux can be configured to confine the scope of (1) and (2):

* To specific (sub)directories (flag `--git-path`)
* To a Git branch other than `master` (flag `--git-branch`)

## Abstracting out file-access: generators and updaters

Flux allows you to declare configuration files to override file operations (1) and (2), by declaring commands which
perform equivalent actions.

The configuration files are formatted in `YAML` and named `.flux.yaml`. They must be located on the Git repository
(more on this later).

A `commandUpdated` `.flux.yaml` file has the following loosely specified format:

```yaml
version: X
commandUpdated:
  generators:
    - command: generator_command1 g1arg1 g1arg2 ...
    - command: generator_command2 g2arg2 g2arg2 ...
  updaters:
    - containerImage:
        command: containerImage_updater_command1 ciu1arg1 ciu1arg2 ...
      policy:
        command: policy_updater_command1 pu1arg1 pu1arg2 ...
    - containerImage:
        command: containerImage_updater_command2 ciu2arg1 ciu2arg2 ...
      policy:
        command: policy_updater_command2 pu2arg1 pu2arg2 ...
```

> **Note:** For a simpler approach to updates, Flux provides a `patchUpdated` configuration file variant.

The file above is versioned (in order to account for future file format changes). Current version is `1`,
which is enforced.

Also, the file contains two generators (declared in the `generators` entry), used to generate manifests and two updaters
(declared in the `updaters` entry), used to update resources in the Git repository.

The generators are meant as an alternative to Flux manifest reads (1). Each updater is split into a `containerImage`
command and a `policy` command, covering the corresponding two types of workload manifest updates mentioned in (2).

> **Note** Update commands operate on policies, rather than annotations. That is for two reasons:
>
> * It is an implementation detail for Kubernetes manifests specifically that policies are represented as annotations.
> * Some configurations (even those for Kubernetes clusters) may encode policies symbolically.

Here is a specific `.flux.yaml` example, declaring a generator and an updater using [Kustomize](https://github.com/kubernetes-sigs/kustomize)
(see [https://github.com/weaveworks/flux-kustomize-example](https://github.com/weaveworks/flux-kustomize-example)
{'fraction_non_alphanumeric': 0.06686638705811863, 'fraction_numerical': 0.009346914319952068, 'mean_word_length': 4.370656370656371, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 13, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3617164', 'n_tokens_mistral': 2148, 'n_tokens_neox': 2064, 'n_words': 1048}