<commit_before># terra-cli

1. [Setup and run](#setup-and-run)
2. [Requirements](#requirements)
    * [Login](#login)
    * [Spend profile access](#spend-profile-access)
    * [External data](#external-data)
3. [Example usage](#example-usage)
4. [Commands description](#commands-description)
    * [Authentication](#authentication)
    * [Server](#server)
    * [Workspace](#workspace)
    * [Resources](#resources)
    * [Data References](#data-references)
    * [Applications](#applications)
    * [Notebooks](#notebooks)
    * [Groups](#groups)
    * [Spend](#spend)
5. [Workspace context for applications](#workspace-context-for-applications)
    * [Reference in a CLI command](#reference-in-a-cli-command)
    * [Reference in file](#reference-in-file)
    * [See all environment variables](#see-all-environment-variables)
6. [Troubleshooting](#troubleshooting)

-----

### Setup and run
From the top-level directory.
```
source tools/local-dev.sh
terra
```

### Requirements
1. Java 11
2. Docker 20.10.2 (Must be running)

#### Login
1. Use a Google account that is not a Google/Verily corporate account.
2. `terra auth login` launches an OAuth flow that pops out a browser window with a warning login
page ("! Google hasn't verified this app"). This shows up because the CLI is not yet a Google-verified
app. Click through the warnings ("Advanced" -> "Go to ... (unsafe)") to complete the login.
3. If the machine where you're running the CLI does not have a browser available to it, then use the
manual login flow by setting the browser flag `terra auth set-browser manual`. See the [Authentication](#authentication)
section below for more details.

#### Spend profile access
In order to spend money (e.g. by creating a project and resources within it) in Terra, you need
access to a billing account via a spend profile. Currently, there is a single spend profile used
by Workspace Manager. Your email needs to either be added as a user of that spend profile or added
to a Terra group that is a user of that spend profile. This needs to be done by someone else with
owner access to that spend profile.

- [Preferred] Add a user to a Terra group that is a user of the spend profile. To also grant permission
to add new members to the group, use `policy=admin` instead.

`terra groups add-user --group=enterprise-pilot-testers --policy=member <EMAIL>`

- Add a user directly to the spend profile. To also grant permission to add new users to the spend profile,
user `policy=owner` instead.
`terra spend enable --policy=user <EMAIL>`

#### External data 
To allow supported applications (i.e. the ones shown by `terra app list`) to read or write data
external to the Terra workspace, you need to give the user's pet service account the appropriate
access. To get the email of the user's pet service account, run `terra gcloud config get-value account`.

### Example usage
The commands below walk through a brief demo of the existing commands.

Fetch the user's credentials.
Check the authentication status to confirm the login was successful.
```
terra auth login
terra auth status
```

Ping the Terra server.
```
terra server status
```

Create an empty directory and change into it.
Create a new Terra workspace (and backing Google project).
Check the current workspace status to confirm it was created successfully.
```
mkdir new-workspace; cd new-workspace
terra workspace create
terra status
```

If you wanted to use an existing Terra workspace, use the mount command instead of create.
```
terra workspace mount eb0753f9-5c45-46b3-b3b4-80b4c7bea248
```

Call the Gcloud CLI tools within the workspace context.
This means the commands are executed against the workspace project and as the current user.
```
terra gcloud config get-value project
terra gsutil ls
terra bq version
```

Enable the Nextflow tool. This will create a sub-directory called `nextflow`.
Run a Nextflow hello world example.
```
terra app enable nextflow
terra nextflow run hello
```

See the list of supported (external) tools.
The CLI runs these tools in a Docker image. Print the image tag that the CLI is currently using.
```
terra app list
terra app get-image
```

### Commands description
```
Usage: terra [COMMAND]
Terra CLI
Commands:
  status     Print details about the current workspace.
  auth       Retrieve and manage user credentials.
  server     Connect to a Terra server.
  workspace  Setup a Terra workspace.
  resources  Manage controlled resources in the workspace.
  data-refs  Reference data in the workspace context.
  app        Run applications in the workspace.
  notebooks  Use AI Notebooks in the workspace.
  groups     Manage groups of users.
  spend      Manage spend profiles.
```

The `status` command prints details about the current workspace and server.

Each sub-group of commands is described in a sub-section below:
- Authentication
- Server
- Workspace
- Resources
- Data References
- Applications
- Notebooks
- Groups
- Spend

#### Authentication
```
Usage: terra auth [COMMAND]
Retrieve and manage user credentials.
Commands:
  status       Print details about the currently authorized account.
  login        Authorize the CLI to access Terra APIs and data with user
                 credentials.
  revoke       Revoke credentials from an account.
  get-browser  Check whether a browser is launched automatically during the
                 login process.
  set-browser  Configure whether a browser is launched automatically during the
                 login process.
```

Only one user can be logged in at a time. To change the active user, revoke the existing credentials and login again.

Login uses the Google OAuth 2.0 installed application [flow](https://developers.google.com/identity/protocols/oauth2/native-app).
If there is a workspace defined in the current context, then logging in also fetches the user's pet SA file for that workspace.<commit_msg>Add random space.<commit_after># terra-cli

1. [Setup and run](#setup-and-run)
2. [Requirements](#requirements)
    * [Login](#login)
    * [Spend profile access](#spend-profile-access)
    * [External data](#external-data)
3. [Example usage](#example-usage)
4. [Commands description](#commands-description)
    * [Authentication](#authentication)
    * [Server](#server)
    * [Workspace](#workspace)
    * [Resources](#resources)
    * [Data References](#data-references)
    * [Applications](#applications)
    * [Notebooks](#notebooks)
    * [Groups](#groups)
    * [Spend](#spend)
5. [Workspace context for applications](#workspace-context-for-applications)
    * [Reference in a CLI command](#reference-in-a-cli-command)
    * [Reference in file](#reference-in-file)
    * [See all environment variables](#see-all-environment-variables)
6. [Troubleshooting](#troubleshooting)

-----

### Setup and run
From the top-level directory.
```
source tools/local-dev.sh
terra
```

### Requirements
1. Java 11
2. Docker 20.10.2 (Must be running)

#### Login
1. Use a Google account that is not a Google/Verily corporate account.
2. `terra auth login` launches an OAuth flow that pops out a browser window with a warning login
page ("! Google hasn't verified this app"). This shows up because the CLI is not yet a Google-verified
app. Click through the warnings ("Advanced" -> "Go to ... (unsafe)") to complete the login.
3. If the machine where you're running the CLI does not have a browser available to it, then use the
manual login flow by setting the browser flag `terra auth set-browser manual`. See the [Authentication](#authentication)
section below for more details.

#### Spend profile access
In order to spend money (e.g. by creating a project and resources within it) in Terra, you need
access to a billing account via a spend profile. Currently, there is a single spend profile used
by Workspace Manager. Your email needs to either be added as a user of that spend profile or added
to a Terra group that is a user of that spend profile. This needs to be done by someone else with
owner access to that spend profile.

- [Preferred] Add a user to a Terra group that is a user of the spend profile. To also grant permission
to add new members to the group, use `policy=admin` instead.

`terra groups add-user --group=enterprise-pilot-testers --policy=member <EMAIL>`

- Add a user directly to the spend profile. To also grant permission to add new users to the spend profile,
user `policy=owner` instead.
`terra spend enable --policy=user <EMAIL>`

#### External data 
To allow supported applications (i.e. the ones shown by `terra app list`) to read or write data
external to the Terra workspace, you need to give the user's pet service account the appropriate
access. To get the email of the user's pet service account, run `terra gcloud config get-value account`.

### Example usage
The commands below walk through a brief demo of the existing commands.

Fetch the user's credentials.
Check the authentication status to confirm the login was successful.
```
terra auth login
terra auth status
```

Ping the Terra server.
```
terra server status
```

Create an empty directory and change into it.
Create a new Terra workspace (and backing Google project).
Check the current workspace status to confirm it was created successfully.
```
mkdir new-workspace; cd new-workspace
terra workspace create
terra status
```

If you wanted to use an existing Terra workspace, use the mount command instead of create.
```
terra workspace mount eb0753f9-5c45-46b3-b3b4-80b4c7bea248
```

Call the Gcloud CLI tools within the workspace context.
This means the commands are executed against the workspace project and as the current user.
```
terra gcloud config get-value project
terra gsutil ls
terra bq version
```

Enable the Nextflow tool. This will create a sub-directory called `nextflow`.
Run a Nextflow hello world example.
```
terra app enable nextflow
terra nextflow run hello
```

See the list of supported (external) tools.
The CLI runs these tools in a Docker image. Print the image tag that the CLI is currently using.
```
terra app list
terra app get-image
```

### Commands description
```
Usage: terra [COMMAND]
Terra CLI
Commands:
  status     Print details about the current workspace.
  auth       Retrieve and manage user credentials.
  server     Connect to a Terra server.
  workspace  Setup a Terra workspace.
  resources  Manage controlled resources in the workspace.
  data-refs  Reference data in the workspace context.
  app        Run applications in the workspace.
  notebooks  Use AI Notebooks in the workspace.
  groups     Manage groups of users.
  spend      Manage spend profiles.
```

The `status` command prints details about the current workspace and server.

Each sub-group of commands is described in a sub-section below:
- Authentication
- Server
- Workspace
- Resources
- Data References
- Applications
- Notebooks
- Groups
- Spend

#### Authentication 
```
Usage: terra auth [COMMAND]
Retrieve and manage user credentials.
Commands:
  status       Print details about the currently authorized account.
  login        Authorize the CLI to access Terra APIs and data with user
                 credentials.
  revoke       Revoke credentials from an account.
  get-browser  Check whether a browser is launched automatically during the
                 login process.
  set-browser  Configure whether a browser is launched automatically during the
                 login process.
```

Only one user can be logged in at a time. To change the active user, revoke the existing credentials and login again.

Login uses the Google OAuth 2.0 installed application [flow](https://developers.google.com/identity/protocols/oauth2/native-app).
If there is a workspace defined in the current context, then logging in also fetches the user's pet SA file for that workspace.
{'fraction_non_alphanumeric': 0.07752398743313238, 'fraction_numerical': 0.006962723953468625, 'mean_word_length': 3.8729830368225073, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 9, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3413818', 'n_tokens_mistral': 3219, 'n_tokens_neox': 3090, 'n_words': 1551}