<commit_before>
Shell Library Extension
=======================

`include 'shell'`

The shell library is an extension available to sink scripts running inside the command-line tool's
environment.

Some commands have options as their first parameter (`opt`) -- see the notes below for a breakdown
of each command's options.

The commands that take files or paths as input can usually accept a list of files too.  For example,
`cat 'foo.txt'` returns a string, but `cat glob '*.txt'` will return a list of strings.

| Function       | Description                                                                    |
|----------------|--------------------------------------------------------------------------------|
| `args`         | Returns the arguments passed into the script from the command line             |
| `cat a`        | Returns the contents of file `a`                                               |
| `cd a`         | Change directory to `a` (home directory if `a` is nil)                         |
| `cp opt, a, b` | Copy files `a` to directory `b`                                                |
| `env a`        | Read an environment variable `a`                                               |
| `exec a, args` | Execute command `a`, using `args` as a list of arguments                       |
| `glob a`       | Converts a glob pattern to a list of files                                     |
| `head a, b`    | Returns the first `a` lines of file `b`                                        |
| `ls opt, a`    | Returns list of files in path `a`                                              |
| `mv opt, a, b` | Move files `a` to `b`                                                          |
| `mkdir opt, a` | Make directory `a`                                                             |
| `pushd a`      | Change directory to `a`, saving previous directory on directory stack          |
| `popd`         | Restore a previous directory from the directory stack                          |
| `pwd`          | Returns current directory                                                      |
| `rm opt, a`    | Remove file `a`                                                                |
| `tail a, b`    | Returns the last `a` lines of file `b`                                         |
| `test a, b`    | Perform test `a` on path `b`                                                   |
| `which a`      | Searches for `a` in system paths; returns full path if found, otherwise nil    |

TODO: more

date, time, sed, grep, touch, set, mktemp, chmod, ln, hostname, rmdir, curl, nc, dirname,
basename, find, whoami, printf, readlink, sort, stat, uniq, uname, sleep

### Options

TODO: this<commit_msg>organizing shell ideas<commit_after>
Shell Library Extension
=======================

`include 'shell'`

The shell library is an extension available to sink scripts running inside the command-line tool's
environment.

| Function       | Description                                                                    |
|----------------|--------------------------------------------------------------------------------|
| `version`      | Verifies a minimum version of the sink executable; returns the current version |
| `args`         | Returns the arguments passed into the script from the command line             |

```
version
exec, which
file: read, copy, write, delete, temp, head, tail, permissions?, touch?
dir: create, delete, change, current, list
path: join, dirname, basename, stat
symlink: create, delete, resolve
date
sleep
env: get, set
args
process: pid, list, kill
user: whoami, id, who
host: hostname, uname
net: ping, listen, wget/curl
glob, find
__filename, __dirname
```
{'fraction_non_alphanumeric': 0.14178100618778586, 'fraction_numerical': 0.0, 'mean_word_length': 1.3194011228945728, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 27, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1944283', 'n_tokens_mistral': 838, 'n_tokens_neox': 812, 'n_words': 385}