<commit_before>SHIELD Genesis Kit
==================

This is a Genesis Kit for the [SHIELD Data Protection System][1],
from [Stark & Wayne][2]. When using [genesis][3] to deploy this kit,
you will get a fully-functional SHIELD deployment for backing up + restoring
your data services.

Quick Start
-----------

To use it, you don't even need to clone this repository!  Just run
the following (using Genesis v2):

```
# create a shield-deployments repo using the latest version of the SHIELD kit
genesis init --kit shield

# create a shield-deployments repo using v1.0.0 of the SHIELD kit
genesis init --kit shield/1.0.0

# create a my-shield-configs repo using the latest version of the SHIELD kit
genesis init --kit shield -d my-shield-configs
```

Once created, refer to the deployment repo's README for information on creating
new environments + deploying them.

Subkits
-------

#### Authentication Backends

When deploying your SHIELD, this kit provides three options for configuring
how users authenticate to SHIELD. One of these three must be specified

- **github-oauth** - Sets up OAuth2 using `github.com` as the OAuth Provider.
  This allows you to give multiple people access to SHIELD, with access based on
  GitHub org membership.
- **cf-oauth** - Sets up OAuth2 using a user-supplied [UAA][4] as the OAuth Provider.
  This allows you to give multiple people access to SHIELD with access based on
  their group membership inside the UAA.
- **http-auth** - Sets up HTTP Basic Authentication and a single user/password
  to be used by SHIELD for authenticating.

Params
------

#### Base Params

There is one required params for SHIELD when deployed with no subkits enabled.

- **params.shield_static_ip** - Choose a static IP from the network in your Cloud Config.
  External SHIELD agents will call home to this IP.

Additionally, the following params can be overridden to customize your installation
if needed:

- **params.installation** - controls the name of the SHIELD installation, as reported
  from inside the SHIELD UI. This defaults to `S.H.I.E.L.D. Alpha`
- **params.shield_disk_pool** - used to define the persistent disk pool that the SHIELD VM will
  be given. This pool must exist in the Cloud Config of the BOSH director that deploys
  SHIELD. This defaults to `shield`.
- **params.shield_vm_type** - used to define the Cloud Config VM type that the SHIELD VM
  will be given. This VM type must exist in the Cloud config of the BOSH director that
  deploys SHIELD. This defaults to `small`, as the SHIELD daemon does not consume many
  resources.
- **params.shield_network** - used to define the Cloud Config network that the SHIELD
  VM will be located on. This network must exist in the Cloud Config of the BOSH director
  that deploys SHIELD. It defaults to `shield`, but typically this can be located
  on a shared-infrastructure network. SHIELD will need to be in a network that has SSH
  access to all of the VMs that have SHIELD agents that will be executing backup jobs.

#### cf-oauth Params

Required Params:

- **UAA Client ID** - In order to validate OAuth attempts, SHIELD needs to authenticate
  to the UAA. This is the client ID of the UAA client that SHIELD will use to communicate
  with the UAA. It will need the `openid,scim.real` scopes. This data will be stored in Vault.
- **UAA Client Secret** - This is the UAA client secret for the above client. This data
  stored in Vault.
- **params.authz_allowed_groups** - A list of UAA groups for authorizing SHIELD users.
  If a user authenticating to SHIELD is in at least one of the defined groups, they are
  allowed into shield.

#### github-oauth Params

Required params:

- **GitHub OAuth Client ID** - In order to validate OAuth attempts, SHIELD needs to authenticate
  to GitHub. When you configure an OAuth integration with GitHub, this will be the `client_id`.
  See https://developer.github.com/v3/oauth/ for more info.
- **GitHub OAuth Client Secret** - This is the GitHub `client_secret` for the above OAuth client.
  This data stored in Vault.
- **params.authz_allowed_groups** - A list of GitHub Organizations for authorizing SHIELD users.
  If a user authenticating to SHIELD is in at least one of the defined orgs, they are
  allowed into shield.

Cloud Config
------------

By default, SHIELD uses the following VM types/networks/disk pools from your
Cloud Config. Feel free to override them in your environment, if you would
rather they use entities already existing in your Cloud Foundry:

```
params:
  shield_network:   shield
  shield_disk_pool: shield # should be at least 1GB
  shield_vm_type:   small # VMs should have at least 1 CPU, and 1GB of memory
```

[1]: https://github.com/starkandwayne/shield<commit_msg>Updated README to reflect new v8 kit changes<commit_after>SHIELD Genesis Kit
==================

This is a Genesis Kit for the [SHIELD Data Protection System][1],
from [<NAME>][2]. When using [genesis][3] to deploy this kit,
you will get a fully-functional SHIELD deployment for backing up + restoring
your data services.

Quick Start
-----------

To use it, you don't even need to clone this repository!  Just run
the following (using Genesis v2):

```
# create a shield-deployments repo using the latest version of the SHIELD kit
genesis init --kit shield

# create a shield-deployments repo using v1.0.0 of the SHIELD kit
genesis init --kit shield/1.0.0

# create a my-shield-configs repo using the latest version of the SHIELD kit
genesis init --kit shield -d my-shield-configs
```

Once created, refer to the deployment repo's README for information on creating
new environments + deploying them.

Subkits
-------

#### Authentication Backends

When deploying your SHIELD, this kit provides three options for configuring
how users authenticate to SHIELD. One of these three must be specified

- **oauth-provider** - Sets up OAuth2 using Github or UAA.
  This allows you to give multiple people access to SHIELD, with access based on GitHub org/team or UAA scim rights.
- **Shield Authentication Backend** - Sets up robust user authentication
system backed by an internal local user database to be used by SHIELD for authentication.

Params
------

#### Base Params

There is one required params for SHIELD when deployed with no subkits enabled.

- **params.shield_static_ip** - Choose a static IP from the network in your Cloud Config.
  External SHIELD agents will call home to this IP.

Additionally, the following params can be overridden to customize your installation
if needed:

- **params.installation** - controls the name of the SHIELD installation, as reported
  from inside the SHIELD UI. This defaults to `S.H.I.E.L.D. Alpha`
- **params.shield_disk_pool** - used to define the persistent disk pool that the SHIELD VM will
  be given. This pool must exist in the Cloud Config of the BOSH director that deploys
  SHIELD. This defaults to `shield`.
- **params.shield_vm_type** - used to define the Cloud Config VM type that the SHIELD VM
  will be given. This VM type must exist in the Cloud config of the BOSH director that
  deploys SHIELD. This defaults to `small`, as the SHIELD daemon does not consume many
  resources.
- **params.shield_network** - used to define the Cloud Config network that the SHIELD
  VM will be located on. This network must exist in the Cloud Config of the BOSH director
  that deploys SHIELD. It defaults to `shield`, but typically this can be located
  on a shared-infrastructure network. SHIELD will need to be in a network that has SSH
  access to all of the VMs that have SHIELD agents that will be executing backup jobs.

#### oauth-provider Params

Required params:

- **authentication** - A shield OAuth specification for configuring auth providers that maps orgs/scim rights to teanats and roles. Note the use of the SYSTEM tenant which grants system roles instead of to a particular tenant.

Example:
```
authentication:
    - name:       Github
      identifier: github
      backend:    github
      properties:
        client_id:     <client-id>
        client_secret: <client-secret>
        mapping:
          - github: starkandwayne  # <-- github org name
            tenant: starkandwayne  # <-- shield tenant name
            rights:
              - team: Owners       # <-- github team name
                role: admin        # <-- shield role name
              - team: Engineering  #   (first match wins)
                role: engineer
              - role: operator     # = (default match)
          - github: starkandwayne
            tenant: SYSTEM
            rights:
              - team: Owners
                role: admin
    - name:       UAA
      identifier: uaa1
      backend:    uaa
      properties:
        client_id:       <client-id>
        client_secret:   <client-secret>
        uaa_endpoint:    https://uaa.shield.10.10.10.10.netip.cc:8443
        skip_verify_tls: true
        mapping:
          - tenant: UAA          # <-- shield tenant name
            rights:
              - scim: uaa.admin  # <-- uaa scim right
                role: admin      # <-- shield role
                                #   (first match wins)
              - scim: cloud_controller.write
                role: engineer
              - role: operator   # = (default match)
          - tenant: UAA Admins
            rights:
              - scim: uaa.admin
                role: admin
```

#### proxy Params

Required params:

- **http_proxy** - Specify the value to use as the HTTP proxy for all connections. For example, HTTP\_PROXY: http://proxy.mycompany.com:8080/.

- **https_proxy** - Specify the value to use as the HTTPS proxy for all connections. For example, HTTPS\_PROXY: https://proxy.mycompany.com:8080/.

- **no_proxy** - Addresses that needs to resolve on the local network and skip the proxy. For example, NO\_PROXY: localhost,.mycompany.com,192.168.0.10:80


Cloud Config
------------

By default, SHIELD uses the following VM types/networks/disk pools from your
Cloud Config. Feel free to override them in your environment, if you would
rather they use entities already existing in your Cloud Foundry:

```
params:
  shield_network:   shield
  shield_disk_pool: shield # should be at least 1GB
  shield_vm_type:   small # VMs should have at least 1 CPU, and 1GB of memory
```

[1]: https://github.com/starkandwayne/shield
{'fraction_non_alphanumeric': 0.07591142055894014, 'fraction_numerical': 0.006382361473745286, 'mean_word_length': 3.065251572327044, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 8, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2156060', 'n_tokens_mistral': 2951, 'n_tokens_neox': 2815, 'n_words': 1377}