<commit_before># List Of StarThinker Command Line Helpers

Developing solutions is easier with helpers that automate common development 
tasks. Here are a few you can [run from the command line](deploy_package.md).  

## List Of Helpers For Users 

- [Execute Google Oauth](../starthinker/auth/helper.py) 
  - Helps create user credentials form client credendtials.
  - User credentials generated by this script are used by several of the helpers in the -u parameter.
  - Minimal example of oAuth.
  - ```
pip install git+https://github.com/google/starthinker
st_auth -h
```

- [CM Report Inspector](../starthinker/task/dcm/helper.py) 
  - Pull CM Report shema, files, and fields.
  - Useful for building the dbm task JSON.
  - Minimal CM API example.
  - ```
pip install git+https://github.com/google/starthinker
st_cm -h
```

- [DV360 Report Inspector](../starthinker/task/dbm/helper.py) 
  - Pull DV360 Report shema, files, and fields.
  - Useful for building the dbm task JSON.
  - Minimal DV360 API example.
  - ```
pip install git+https://github.com/google/starthinker
st_dv360 -h
```

- [DV360 LineItem Inspector ( Legacy API )](../starthinker/task/lineitem/helper.py) 
  - Pull DV360 LineItems data from legacy API.
  - Useful for building the lineitem task JSON.
  - Minimal DV360 Line Item API example.
  - ```
pip install git+https://github.com/google/starthinker
st_dv360_lineitem_legacy -h
```

- [DV360 LineItem Download ( Beta API )](../starthinker/task/dv360_beta/helper.py) 
  - Pull DV360 Beta API LineItems.
  - Minimal DV360 Beta API example.
  - ```
pip install git+https://github.com/google/starthinker
st_dv360_lineitem -h
```

- [Call Any Google API Endpoint](../starthinker/task/google_api/helper.py) 
  - Quickly see the results of any Google API Endpoint, great for debugging format and access.
  - ```
pip install git+https://github.com/google/starthinker
st_google_api -h
```

- [Verify JSON is Valid](../starthinker/script/helper.py) 
  - StarThinker JSON allows newlines for queries etc.
  - This utility checks and correctly prints error locations.
  - ```
pip install git+https://github.com/google/starthinker
st_json -h
```

- [Pull BigQuery Table Schema](../starthinker/task/bigquery/helper.py) 
  - Quickly pull table schema and for a recipe.
  - ```
pip install git+https://github.com/google/starthinker
st_bigquery -h
```

- [Send An Email Template](../starthinker/task/newsletter/helper.py) 
  - Quickly send a nicely [formatted email](../starthinker/util/email/template.py), including tables, to anyone.
  - Useful for automating reporting emails, see [sample.json](../starthinker/task/newlsetter/sample.json) template.
  - ```
pip install git+https://github.com/google/starthinker
st_newsletter -h
```

- [Run Automated Tests](../tests/helper.py) 
  - Execute one or all automated tests within StarThinker.
  - [Developer deployment](deploy_developer.md) only.



## Running Helpers For Developers
For those of you with a [developer deployment](deploy_developer.md) of the repository, you can customize and 
for example run a helper using:

```
source starthinker_assets/development.sh
python starthinker/task/dbm/helper.py --list -u $STARTHINKER_USER -c $STARTHINKER_CLIENT
```

The most common development environment parameters for helpers are:

- -u $STARTHINKER_USER - user credentials for accessing API endpoints.
- -c $STARTHINKER_CLIENT - client creddentials for authenticating, not needed if user credentials already exist.
- -s $STARTHINKER_SERVICE - service credentials for writing data.
- -p $STARTHINKER_PROJECT - cloud project to bill the data transfer to.

---
&copy; 2020 Google LLC - Apache License, Version 2.0
<commit_msg>Updating package deployment for command line.<commit_after># List Of StarThinker Command Line Helpers

Developing solutions is easier with helpers that automate common development 
tasks. Here are a few you can [run from the command line](deploy_package.md).  

## List Of Helpers For Users 

- [Execute Google Oauth](../starthinker/auth/helper.py) 
  - Helps create user credentials form client credendtials.
  - User credentials generated by this script are used by several of the helpers in the -u parameter.
  - Minimal example of oAuth.
```
pip install git+https://github.com/google/starthinker
st_auth -h
```

- [CM Report Inspector](../starthinker/task/dcm/helper.py) 
  - Pull CM Report shema, files, and fields.
  - Useful for building the dbm task JSON.
  - Minimal CM API example.
```
pip install git+https://github.com/google/starthinker
st_cm -h
```

- [DV360 Report Inspector](../starthinker/task/dbm/helper.py) 
  - Pull DV360 Report shema, files, and fields.
  - Useful for building the dbm task JSON.
  - Minimal DV360 API example.
```
pip install git+https://github.com/google/starthinker
st_dv360 -h
```

- [DV360 LineItem Inspector ( Legacy API )](../starthinker/task/lineitem/helper.py) 
  - Pull DV360 LineItems data from legacy API.
  - Useful for building the lineitem task JSON.
  - Minimal DV360 Line Item API example.
  - ```
pip install git+https://github.com/google/starthinker
st_dv360_lineitem_legacy -h
```

- [DV360 LineItem Download ( Beta API )](../starthinker/task/dv360_beta/helper.py) 
  - Pull DV360 Beta API LineItems.
  - Minimal DV360 Beta API example.
  - ```
pip install git+https://github.com/google/starthinker
st_dv360_lineitem -h
```

- [Call Any Google API Endpoint](../starthinker/task/google_api/helper.py) 
  - Quickly see the results of any Google API Endpoint, great for debugging format and access.
  - ```
pip install git+https://github.com/google/starthinker
st_google_api -h
```

- [Verify JSON is Valid](../starthinker/script/helper.py) 
  - StarThinker JSON allows newlines for queries etc.
  - This utility checks and correctly prints error locations.
  - ```
pip install git+https://github.com/google/starthinker
st_json -h
```

- [Pull BigQuery Table Schema](../starthinker/task/bigquery/helper.py) 
  - Quickly pull table schema and for a recipe.
  - ```
pip install git+https://github.com/google/starthinker
st_bigquery -h
```

- [Send An Email Template](../starthinker/task/newsletter/helper.py) 
  - Quickly send a nicely [formatted email](../starthinker/util/email/template.py), including tables, to anyone.
  - Useful for automating reporting emails, see [sample.json](../starthinker/task/newlsetter/sample.json) template.
  - ```
pip install git+https://github.com/google/starthinker
st_newsletter -h
```

- [Run Automated Tests](../tests/helper.py) 
  - Execute one or all automated tests within StarThinker.
  - [Developer deployment](deploy_developer.md) only.



## Running Helpers For Developers
For those of you with a [developer deployment](deploy_developer.md) of the repository, you can customize and 
for example run a helper using:

```
source starthinker_assets/development.sh
python starthinker/task/dbm/helper.py --list -u $STARTHINKER_USER -c $STARTHINKER_CLIENT
```

The most common development environment parameters for helpers are:

- -u $STARTHINKER_USER - user credentials for accessing API endpoints.
- -c $STARTHINKER_CLIENT - client creddentials for authenticating, not needed if user credentials already exist.
- -s $STARTHINKER_SERVICE - service credentials for writing data.
- -p $STARTHINKER_PROJECT - cloud project to bill the data transfer to.

---
&copy; 2020 Google LLC - Apache License, Version 2.0

{'fraction_non_alphanumeric': 0.10969664138678223, 'fraction_numerical': 0.01218851570964247, 'mean_word_length': 3.773755656108597, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 18, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4292999', 'n_tokens_mistral': 2494, 'n_tokens_neox': 2261, 'n_words': 745}