<commit_before>---
title: "Configuration"
date: 2018-04-24T18:26:56-07:00
draft: false
weight: 40
---

:toc:
Latest Release: 3.4.0 {docdate}

== Overview

This document describes how to configure the operator beyond the default configurations in addition to detailing what the configuration settings mean.

== Openshift Container Platform

To run the Operator on Openshift Container Platform note the following requirements -

 * Openshift Container Platform 3.7 or greater is required due to the dependence on Custom Resource Definitions.
 * The `CO_CMD` environment variable should be set to `oc` when operating in an Openshift environment.

== Security Configuration

=== Kube RBAC

The `cluster-rbac.yaml` file is executed a single time when installing
the Operator.  This file, executed by a Kubernetes user with cluster-admin
priviledges, does the following:

 * Creates Customer Resource Definitions
 * Grants `get` access to Kube Node resources to the postgres-operator
   service account.

The `rbac.yaml` file is also executed a single time when installing
the Operator.  This file creates Role scoped privileges which are
granted to the postgres-operator service account.  The postgres-operator
service account is used by the *apiserver* and *postgres-operator* containers
to access Kubernetes resources.

Both of these RBAC files are executed by the `deploy/install-rbac.sh`
script. It can also be installed through running `make installrbac` in the
$CCPROOT directory.

[WARNING]
====
The CO_NAMESPACE environment variable determines the namespace
that is used within the deployment of the operator.  If you
are deploying to the *demo* namespace, the following
should setting should be defined in your .bashrc:
`export CO_NAMESPACE=demo`
====

{{% notice tip %}}
See link:https://kubernetes.io/docs/admin/authorization/rbac/[here] for more
details on how to enable RBAC roles and modify the scope of the permissions
to suit your needs.
{{% /notice %}}

=== Basic Authentication

Basic authentication between the host and the apiserver is required. It will
be necessary to configure the pgo client to specify a basic authentication
username and password through the creation a file in the user's home directory
named `.pgouser`. It will look similar to this, and contain only a single line -
....
username:password
....

The above excerpt specifies a username of *username* and a password of *password*.
These values will be read by the *pgo* client and passed to the *apiserver* on each
REST API call.

For the *apiserver*, a list of usernames and passwords is specified in the
*apiserver-conf-secret* Secret.  The values specified in a deployment are found in
the following location -
....
$COROOT/conf/apiserver/pgouser
....

The sample configuration for `pgouser` is as follows -
....
username:password:<PASSWORD>
testuser:testpass:<PASSWORD>
readonlyuser:testpass:pg<PASSWORD>ader
....

Modify these values to be unique to your environment.

If the username and password passed by clients to the *apiserver* do
not match, the REST call will fail and a log message will be produced
in the *apiserver* container log. The client will receive a 401 HTTP
status code if they are not able to authenticate.

If the `pgouser` file is not found in the *home* directory of the pgo user
then the next searched location is `/etc/pgo/pgouser`. If the file is not
found in either of the locations, the pgo client searches for the existence
of a `PGOUSER` environment variable in order to locate a path to the basic
authentication file.

Basic authentication can be entirely disabled by setting the BasicAuth
setting in the `pgo.yaml` configuration file to `false`.

=== Configure TLS

TLS is used to secure communications to the apiserver. Sample keys and
certifications that can be used by TLS are found here -
....
$COROOT/conf/apiserver/server.crt
$COROOT/conf/apiserver/server.key
....

If you want to generate your own keys, you can use the script found in -
....
$COROOT/bin/make-certs.sh
....

The *pgo* client is required to use keys to connect to the *apiserver*.
Specify the keys for pgo by setting the following environment variables -
....
export PGO_CA_CERT=$COROOT/conf/apiserver/server.crt
export PGO_CLIENT_CERT=$COROOT/conf/apiserver/server.crt
export PGO_CLIENT_KEY=$COROOT/conf/apiserver/server.key
....

The sample server keys are used as the client keys; adjust to suit
security requirements.

For the *apiserver TLS configuration*, the keys are included in the
*apiserver-conf-secret* Secret when the apiserver is deployed. See the
`$COROOT/deploy/deploy.sh script` which is where the secret is created.

The apiserver listens on port 8443 (e.g. https://postgres-operator:8443)
by default.

You can set `InsecureSkipVerify` to *true* by setting the `NO_TLS_VERIFY`
environment variable in the `deployment.json` file to *true*. By default
this value is set to *false* if you do not specify a value.

=== pgo RBAC

The pgo command line utility talks to the apiserver REST API instead of
the Kubernetes API. It is therefore necessary for the pgo client to make
use of RBAC configuration.

Starting in Release 3.0, the */conf/apiserver/pgorole* is used to define some sample pgo roles, *pgadmin* and *pgoreader*.

These roles are meant as examples that you can configure to suit security
requirements as necessary. The *pgadmin* role grants a user authorization to
all pgo commands. The *pgoreader* only grants access to pgo commands that
display information such as `pgo show cluster`.

The `pgorole` file is read at start up time when the operator is deployed to
the Kubernetes cluster.

Also, the `pgouser` file now includes the role that is assigned to a specific
user as follows -
....
username:password:pgoadmin
testuser:testpass:pgoadmin
readonlyuser:testpass:pgoreader
....

The following list shows the current complete list of possible pgo
permissions -

.pgo Permissions
[width="60%",frame="topbot",options="header"]
|======================
|Permission | Description
|ShowSecrets   | allow *pgo show user*
|ShowCluster   | allow *pgo show cluster*
|CreateCluster | allow *pgo create cluster*
|TestCluster   | allow *pgo test mycluster*
|ShowBackup    | allow *pgo show backup*
|CreateBackup  | allow *pgo backup mycluster*
|DeleteBackup  | allow *pgo delete backup mycluster*
|Label         | allow *pgo label*
|Load          | allow *pgo load*
|CreatePolicy  | allow *pgo create policy*
|DeletePolicy  | allow *pgo delete policy*
|ShowPolicy    | allow *pgo show policy*
|ApplyPolicy   | allow *pgo apply policy*
|ShowPVC       | allow *pgo show pvc*
|CreateUpgrade | allow *pgo upgrade*
|ShowUpgrade   | allow *pgo show upgrade*
|DeleteUpgrade | allow *pgo delete upgrade*
|CreateUser    | allow *pgo create user*
|CreateFailover| allow *pgo failover*
|ShowConfig    | allow *pgo show config*
|User          | allow *pgo user*
|Version       | allow *pgo version*
|======================

If the user is unauthorized for a pgo command, the user will
get back this response -
....
FATA[0000] Authentication Failed: 40
....


=== REST API Configuration

The postgres-operator pod includes the apiserver which is a REST API that pgo
users are able to communicate with.

The apiserver uses the following configuration files found in `$COROOT/conf/apiserver`
to determine how the Operator will provision PostgreSQL containers -
....
$COROOT/conf/apiserver/pgo.yaml
$COROOT/conf/apiserver/pgo.lspvc-template.json
$COROOT/conf/apiserver/pgo.load-template.json
....

Note that the default pgo.yaml file assumes you are going to use *HostPath* Persistent
Volumes for your storage configuration. It will be necessary to adjust this file for NFS
or other storage configurations. Some examples of how are listed in the manual installation
document.

The version of PostgreSQL container the Operator will deploy is determined by the *CCPImageTag*
setting in the `$COROOT/conf/apiserver/pgo.yaml` configuration file. By default, this value is
set to the latest release of the Crunchy Container Suite.

The default pgo.yaml configuration file, included in `$COROOT/conf/apiserver/pgo.yaml`,
looks like this -

[source,yaml]
....
BasicAuth:  true
Cluster:
  PrimaryNodeLabel:  
  ReplicaNodeLabel:  
  CCPImageTag:  centos7-10.5-2.1.0
  Metrics:  false
  Badger:  false
  Port:  5432
  User:  testuser
  Database:  userdb
  PasswordAgeDays:  60
  PasswordLength:  8
  Strategy:  1
  Replicas:  0
  ArchiveMode:  false
  ArchiveTimeout:  60
  ServiceType:  ClusterIP
PrimaryStorage: storage1
BackupStorage: storage1
ReplicaStorage: storage1
Storage:
  storage1:
    AccessMode:  ReadWriteMany
    Size:  200M
    StorageType:  create
  storage2:
    AccessMode:  ReadWriteMany
    Size:  333M
    StorageType:  create
  storage3:
    AccessMode:  ReadWriteMany
    Size:  440M
    StorageType:  create
DefaultContainerResource: small
ContainerResources:
  small:
    RequestsMemory:  2Gi
    RequestsCPU:  0.5
    LimitsMemory:  2Gi
    LimitsCPU:  1.0
  large:
    RequestsMemory:  8Gi
    RequestsCPU:  2.0
    LimitsMemory:  12Gi
    LimitsCPU:  4.0
Pgo:
  Audit:  false
  LSPVCTemplate:  /config/pgo.lspvc-template.json
  CSVLoadTemplate:  /config/pgo.load-template.json
  COImagePrefix:  crunchydata
  COImageTag:  centos7-3.4.0
....

Values in the pgo configuration file have the following meaning:

.pgo Configuration File Definitions
[width="90%",cols="m,2",frame="topbot",options="header"]
|======================
|Setting | Definition
|BasicAuth        | if set to *true* will enable Basic Authentication
|Cluster.PrimaryNodeLabel        |newly created primary deployments will specify this node label if specified, unless you override it using the --node-label command line flag, if not set, no node label is specifed
|Cluster.ReplicaNodeLabel        |newly created replica deployments will specify this node label if specified, unless you override it using the --node-label command line flag, if not set, no node label is specifed
|Cluster.CCPImageTag        |newly created containers will be based on this image version (e.g. centos7-10.4-1.8.3), unless you override it using the --ccp-image-tag command line flag
|Cluster.Port        | the PostgreSQL port to use for new containers (e.g. 5432)
|Cluster.User        | the PostgreSQL normal user name
|Cluster.Strategy        | sets the deployment strategy to be used for deploying a cluster, currently there is only strategy *1*
|Cluster.Replicas        | the number of cluster replicas to create for newly created clusters
|Cluster.Metrics        | boolean, if set to true will cause each new cluster to include crunchy-collect as a sidecar container for metrics collection, if set to false (default), users can still add metrics on a cluster-by-cluster basis using the pgo command flag --metrics
|Cluster.Badger        | boolean, if set to true will cause each new cluster to include crunchy-pgbadger as a sidecar container for static log analysis, if set to false (default), users can still add pgbadger on a cluster-by-cluster basis using the pgo create cluster command flag --pgbadger
|Cluster.Policies        | optional, list of policies to apply to a newly created cluster, comma separated, must be valid policies in the catalog
|Cluster.PasswordAgeDays        | optional, if set, will set the VALID UNTIL date on passwords to this many days in the future when creating users or setting passwords, defaults to 60 days
|Cluster.PasswordLength        | optional, if set, will determine the password length used when creating passwords, defaults to 8
|Cluster.ArchiveMode        | optional, if set to true will enable archive logging for all clusters created, default is false.
|Cluster.ArchiveTimeout        | optional, if set, will determine the archive timeout setting used when ArchiveMode is true, defaults to 60 seconds
|Cluster.ServiceType        | optional, if set, will determine the service type used when creating primary or replica services, defaults to ClusterIP if not set, can be overridden by the user on the command line as well
|Cluster.Backrest        | optional, if set, will cause clusters to have the pgbackrest volume PVC provisioned during cluster creation
|Cluster.Autofail        | optional, if set, will cause clusters to be checked for auto failover in the event of a non-Ready status
|PrimaryStorage    |required, the value of the storage configuration to use for the primary PostgreSQL deployment
|BackupStorage    |required, the value of the storage configuration to use for backups, including the storage for pgbackrest repo volumes
|ReplicaStorage    |required, the value of the storage configuration to use for the replica PostgreSQL deployments
|Storage.storage1.StorageClass        |for a dynamic storage type, you can specify the storage class used for storage provisioning(e.g. standard, gold, fast)
|Storage.storage1.AccessMode        |the access mode for new PVCs (e.g. ReadWriteMany, ReadWriteOnce, ReadOnlyMany). See below for descriptions of these.
|Storage.storage1.Size        |the size to use when creating new PVCs (e.g. 100M, 1Gi)
|Storage.storage1.StorageType        |supported values are either *dynamic*,  *create*,  if not supplied, *create* is used
|Storage.storage1.Fsgroup        | optional, if set, will cause a *SecurityContext* and *fsGroup* attributes to be added to generated Pod and Deployment definitions
|Storage.storage1.SupplementalGroups        | optional, if set, will cause a SecurityContext to be added to generated Pod and Deployment definitions
|Storage.storage1.MatchLabels        | optional, if set, will cause the PVC to add a *matchlabels* selector in order to match a PV, only useful when the StorageType is *create*, when specified a label of *name=clustername* is added to the PVC as a match criteria
|DefaultContainerResource    |optional, the value of the container resources configuration to use for all database containers, if not set, no resource limits or requests are added on the database container
|ContainerResources.small.RequestsMemory        | request size of memory in bytes
|ContainerResources.small.RequestsCPU        | request size of CPU cores
|ContainerResources.small.LimitsMemory        | request size of memory in bytes
|ContainerResources.small.LimitsCPU        | request size of CPU cores
|ContainerResources.large.RequestsMemory        | request size of memory in bytes<commit_msg>update docs for resource config changes<commit_after>---
title: "Configuration"
date: 2018-04-24T18:26:56-07:00
draft: false
weight: 40
---

:toc:
Latest Release: 3.4.0 {docdate}

== Overview

This document describes how to configure the operator beyond the default configurations in addition to detailing what the configuration settings mean.

== Openshift Container Platform

To run the Operator on Openshift Container Platform note the following requirements -

 * Openshift Container Platform 3.7 or greater is required due to the dependence on Custom Resource Definitions.
 * The `CO_CMD` environment variable should be set to `oc` when operating in an Openshift environment.

== Security Configuration

=== Kube RBAC

The `cluster-rbac.yaml` file is executed a single time when installing
the Operator.  This file, executed by a Kubernetes user with cluster-admin
priviledges, does the following:

 * Creates Customer Resource Definitions
 * Grants `get` access to Kube Node resources to the postgres-operator
   service account.

The `rbac.yaml` file is also executed a single time when installing
the Operator.  This file creates Role scoped privileges which are
granted to the postgres-operator service account.  The postgres-operator
service account is used by the *apiserver* and *postgres-operator* containers
to access Kubernetes resources.

Both of these RBAC files are executed by the `deploy/install-rbac.sh`
script. It can also be installed through running `make installrbac` in the
$CCPROOT directory.

[WARNING]
====
The CO_NAMESPACE environment variable determines the namespace
that is used within the deployment of the operator.  If you
are deploying to the *demo* namespace, the following
should setting should be defined in your .bashrc:
`export CO_NAMESPACE=demo`
====

{{% notice tip %}}
See link:https://kubernetes.io/docs/admin/authorization/rbac/[here] for more
details on how to enable RBAC roles and modify the scope of the permissions
to suit your needs.
{{% /notice %}}

=== Basic Authentication

Basic authentication between the host and the apiserver is required. It will
be necessary to configure the pgo client to specify a basic authentication
username and password through the creation a file in the user's home directory
named `.pgouser`. It will look similar to this, and contain only a single line -
....
username:password
....

The above excerpt specifies a username of *username* and a password of *password*.
These values will be read by the *pgo* client and passed to the *apiserver* on each
REST API call.

For the *apiserver*, a list of usernames and passwords is specified in the
*apiserver-conf-secret* Secret.  The values specified in a deployment are found in
the following location -
....
$COROOT/conf/apiserver/pgouser
....

The sample configuration for `pgouser` is as follows -
....
username:password:<PASSWORD>
testuser:testpass:<PASSWORD>
readonlyuser:testpass:<PASSWORD>
....

Modify these values to be unique to your environment.

If the username and password passed by clients to the *apiserver* do
not match, the REST call will fail and a log message will be produced
in the *apiserver* container log. The client will receive a 401 HTTP
status code if they are not able to authenticate.

If the `pgouser` file is not found in the *home* directory of the pgo user
then the next searched location is `/etc/pgo/pgouser`. If the file is not
found in either of the locations, the pgo client searches for the existence
of a `PGOUSER` environment variable in order to locate a path to the basic
authentication file.

Basic authentication can be entirely disabled by setting the BasicAuth
setting in the `pgo.yaml` configuration file to `false`.

=== Configure TLS

TLS is used to secure communications to the apiserver. Sample keys and
certifications that can be used by TLS are found here -
....
$COROOT/conf/apiserver/server.crt
$COROOT/conf/apiserver/server.key
....

If you want to generate your own keys, you can use the script found in -
....
$COROOT/bin/make-certs.sh
....

The *pgo* client is required to use keys to connect to the *apiserver*.
Specify the keys for pgo by setting the following environment variables -
....
export PGO_CA_CERT=$COROOT/conf/apiserver/server.crt
export PGO_CLIENT_CERT=$COROOT/conf/apiserver/server.crt
export PGO_CLIENT_KEY=$COROOT/conf/apiserver/server.key
....

The sample server keys are used as the client keys; adjust to suit
security requirements.

For the *apiserver TLS configuration*, the keys are included in the
*apiserver-conf-secret* Secret when the apiserver is deployed. See the
`$COROOT/deploy/deploy.sh script` which is where the secret is created.

The apiserver listens on port 8443 (e.g. https://postgres-operator:8443)
by default.

You can set `InsecureSkipVerify` to *true* by setting the `NO_TLS_VERIFY`
environment variable in the `deployment.json` file to *true*. By default
this value is set to *false* if you do not specify a value.

=== pgo RBAC

The pgo command line utility talks to the apiserver REST API instead of
the Kubernetes API. It is therefore necessary for the pgo client to make
use of RBAC configuration.

Starting in Release 3.0, the */conf/apiserver/pgorole* is used to define some sample pgo roles, *pgadmin* and *pgoreader*.

These roles are meant as examples that you can configure to suit security
requirements as necessary. The *pgadmin* role grants a user authorization to
all pgo commands. The *pgoreader* only grants access to pgo commands that
display information such as `pgo show cluster`.

The `pgorole` file is read at start up time when the operator is deployed to
the Kubernetes cluster.

Also, the `pgouser` file now includes the role that is assigned to a specific
user as follows -
....
username:password:<PASSWORD>
testuser:testpass:pgoadmin
readonlyuser:testpass:pgoreader
....

The following list shows the current complete list of possible pgo
permissions -

.pgo Permissions
[width="60%",frame="topbot",options="header"]
|======================
|Permission | Description
|ShowSecrets   | allow *pgo show user*
|ShowCluster   | allow *pgo show cluster*
|CreateCluster | allow *pgo create cluster*
|TestCluster   | allow *pgo test mycluster*
|ShowBackup    | allow *pgo show backup*
|CreateBackup  | allow *pgo backup mycluster*
|DeleteBackup  | allow *pgo delete backup mycluster*
|Label         | allow *pgo label*
|Load          | allow *pgo load*
|CreatePolicy  | allow *pgo create policy*
|DeletePolicy  | allow *pgo delete policy*
|ShowPolicy    | allow *pgo show policy*
|ApplyPolicy   | allow *pgo apply policy*
|ShowPVC       | allow *pgo show pvc*
|CreateUpgrade | allow *pgo upgrade*
|ShowUpgrade   | allow *pgo show upgrade*
|DeleteUpgrade | allow *pgo delete upgrade*
|CreateUser    | allow *pgo create user*
|CreateFailover| allow *pgo failover*
|ShowConfig    | allow *pgo show config*
|User          | allow *pgo user*
|Version       | allow *pgo version*
|======================

If the user is unauthorized for a pgo command, the user will
get back this response -
....
FATA[0000] Authentication Failed: 40
....


=== REST API Configuration

The postgres-operator pod includes the apiserver which is a REST API that pgo
users are able to communicate with.

The apiserver uses the following configuration files found in `$COROOT/conf/apiserver`
to determine how the Operator will provision PostgreSQL containers -
....
$COROOT/conf/apiserver/pgo.yaml
$COROOT/conf/apiserver/pgo.lspvc-template.json
$COROOT/conf/apiserver/pgo.load-template.json
....

Note that the default pgo.yaml file assumes you are going to use *HostPath* Persistent
Volumes for your storage configuration. It will be necessary to adjust this file for NFS
or other storage configurations. Some examples of how are listed in the manual installation
document.

The version of PostgreSQL container the Operator will deploy is determined by the *CCPImageTag*
setting in the `$COROOT/conf/apiserver/pgo.yaml` configuration file. By default, this value is
set to the latest release of the Crunchy Container Suite.

The default pgo.yaml configuration file, included in `$COROOT/conf/apiserver/pgo.yaml`,
looks like this -

[source,yaml]
....
BasicAuth:  true
Cluster:
  PrimaryNodeLabel:  
  ReplicaNodeLabel:  
  CCPImageTag:  centos7-10.5-2.1.0
  Metrics:  false
  Badger:  false
  Port:  5432
  User:  testuser
  Database:  userdb
  PasswordAgeDays:  60
  PasswordLength:  8
  Strategy:  1
  Replicas:  0
  ArchiveMode:  false
  ArchiveTimeout:  60
  ServiceType:  ClusterIP
PrimaryStorage: storage1
BackupStorage: storage1
ReplicaStorage: storage1
Storage:
  storage1:
    AccessMode:  ReadWriteMany
    Size:  200M
    StorageType:  create
  storage2:
    AccessMode:  ReadWriteMany
    Size:  333M
    StorageType:  create
  storage3:
    AccessMode:  ReadWriteMany
    Size:  440M
    StorageType:  create
DefaultContainerResource: small
DefaultLoadResources:
DefaultLspvcResources:
DefaultRmdataResources:
DefaultBackupResources:
DefaultPgbouncerResources:
DefaultPgpoolResources:
ContainerResources:
  small:
    RequestsMemory:  2Gi
    RequestsCPU:  0.5
    LimitsMemory:  2Gi
    LimitsCPU:  1.0
  large:
    RequestsMemory:  8Gi
    RequestsCPU:  2.0
    LimitsMemory:  12Gi
    LimitsCPU:  4.0
Pgo:
  Audit:  false
  LSPVCTemplate:  /config/pgo.lspvc-template.json
  CSVLoadTemplate:  /config/pgo.load-template.json
  COImagePrefix:  crunchydata
  COImageTag:  centos7-3.4.0
....

Values in the pgo configuration file have the following meaning:

.pgo Configuration File Definitions
[width="90%",cols="m,2",frame="topbot",options="header"]
|======================
|Setting | Definition
|BasicAuth        | if set to *true* will enable Basic Authentication
|Cluster.PrimaryNodeLabel        |newly created primary deployments will specify this node label if specified, unless you override it using the --node-label command line flag, if not set, no node label is specifed
|Cluster.ReplicaNodeLabel        |newly created replica deployments will specify this node label if specified, unless you override it using the --node-label command line flag, if not set, no node label is specifed
|Cluster.CCPImageTag        |newly created containers will be based on this image version (e.g. centos7-10.4-1.8.3), unless you override it using the --ccp-image-tag command line flag
|Cluster.Port        | the PostgreSQL port to use for new containers (e.g. 5432)
|Cluster.User        | the PostgreSQL normal user name
|Cluster.Strategy        | sets the deployment strategy to be used for deploying a cluster, currently there is only strategy *1*
|Cluster.Replicas        | the number of cluster replicas to create for newly created clusters
|Cluster.Metrics        | boolean, if set to true will cause each new cluster to include crunchy-collect as a sidecar container for metrics collection, if set to false (default), users can still add metrics on a cluster-by-cluster basis using the pgo command flag --metrics
|Cluster.Badger        | boolean, if set to true will cause each new cluster to include crunchy-pgbadger as a sidecar container for static log analysis, if set to false (default), users can still add pgbadger on a cluster-by-cluster basis using the pgo create cluster command flag --pgbadger
|Cluster.Policies        | optional, list of policies to apply to a newly created cluster, comma separated, must be valid policies in the catalog
|Cluster.PasswordAgeDays        | optional, if set, will set the VALID UNTIL date on passwords to this many days in the future when creating users or setting passwords, defaults to 60 days
|Cluster.PasswordLength        | optional, if set, will determine the password length used when creating passwords, defaults to 8
|Cluster.ArchiveMode        | optional, if set to true will enable archive logging for all clusters created, default is false.
|Cluster.ArchiveTimeout        | optional, if set, will determine the archive timeout setting used when ArchiveMode is true, defaults to 60 seconds
|Cluster.ServiceType        | optional, if set, will determine the service type used when creating primary or replica services, defaults to ClusterIP if not set, can be overridden by the user on the command line as well
|Cluster.Backrest        | optional, if set, will cause clusters to have the pgbackrest volume PVC provisioned during cluster creation
|Cluster.Autofail        | optional, if set, will cause clusters to be checked for auto failover in the event of a non-Ready status
|PrimaryStorage    |required, the value of the storage configuration to use for the primary PostgreSQL deployment
|BackupStorage    |required, the value of the storage configuration to use for backups, including the storage for pgbackrest repo volumes
|ReplicaStorage    |required, the value of the storage configuration to use for the replica PostgreSQL deployments
|Storage.storage1.StorageClass        |for a dynamic storage type, you can specify the storage class used for storage provisioning(e.g. standard, gold, fast)
|Storage.storage1.AccessMode        |the access mode for new PVCs (e.g. ReadWriteMany, ReadWriteOnce, ReadOnlyMany). See below for descriptions of these.
|Storage.storage1.Size        |the size to use when creating new PVCs (e.g. 100M, 1Gi)
|Storage.storage1.StorageType        |supported values are either *dynamic*,  *create*,  if not supplied, *create* is used
|Storage.storage1.Fsgroup        | optional, if set, will cause a *SecurityContext* and *fsGroup* attributes to be added to generated Pod and Deployment definitions
|Storage.storage1.SupplementalGroups        | optional, if set, will cause a SecurityContext to be added to generated Pod and Deployment definitions
|Storage.storage1.MatchLabels        | optional, if set, will cause the PVC to add a *matchlabels* selector in order to match a PV, only useful when the StorageType is *create*, when specified a label of *name=clustername* is added to the PVC as a match criteria
|DefaultContainerResource    |optional, the value of the container resources configuration to use for all database containers, if not set, no resource limits or requests are added on the database container
|DefaultLoadResource    |optional, the value of the container resources configuration to use for pgo-load containers, if not set, no resource limits or requests are added on the database container
|DefaultLspvcResource    |optional, the value of the container resources configuration to use for pgo-lspvc containers, if not set, no resource limits or requests are added on the database container
|DefaultRmdataResource    |optional, the value of the container resources configuration to use for pgo-rmdata containers, if not set, no resource limits or requests are added on the database container
|DefaultBackupResource    |optional, the value of the container resources configuration to use for crunchy-backup containers, if not set, no resource limits or requests are added on the database container
|DefaultPgbouncerResource    |optional, the value of the container resources configuration to use for crunchy-pgbouncer containers, if not set, no resource limits or requests are added on the database container
|DefaultPgpoolResource    |optional, the value of the container resources configuration to use for crunchy-pgpool containers, if not set, no resource limits or requests are added on the database container
|ContainerResources.small.RequestsMemory        | request size of memory in bytes
|ContainerResources.small.RequestsCPU        | request size of CPU cores
|ContainerResources.small.LimitsMemory        | request size of memory in bytes
|ContainerResources.small.LimitsCPU        | request size of CPU cores
|ContainerResources.large.RequestsMemory        | request size of memory in bytes
{'fraction_non_alphanumeric': 0.06772908366533864, 'fraction_numerical': 0.008432934926958832, 'mean_word_length': 4.207641770401106, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2756729', 'n_tokens_mistral': 8358, 'n_tokens_neox': 8049, 'n_words': 3842}