<commit_before># baltar
A few small utilities for working with tarballs and http. Because you need tarballs over HTTP like:

![](https://media.giphy.com/media/52HjuHsfVO69q/giphy.gif)

## Usage

### 1. `push` and `pull` tarballs over the network

##### `baltar.pull(opts, callback)`

Makes a request to `opts.url` and unpacks it to `opts.path`. `baltar.pull` is the only method which accepts a callback so that it pass back all of the entries from the `tar.Extract` stream. The end of pipechain is also returned for future stream operations (if desired).

- `opts.url`: {string} Location of the receiver.
- `opts.headers`: {Object} HTTP headers to send.
- `opts.method`: {string} HTTP Method to send.
- `opts.path`: {string} Directory or file to unpack to.
- `opts.tarball`: {string} **Optional** Path to save tarball to.
- `returns`: {tar.Extract} Extraction stream for the pulled tarball.

##### `baltar.push(opts)`

- `opts.path`: Directory or file to pack
- `opts.ignoreFiles`: Extra ignore files to parse
- `opts.url`: {string} Location of the receiver.
- `opts.headers`: {Object} HTTP headers to send.
- `opts.method`: {string} HTTP Method to send.
- `returns`: {Stream} HTTP request stream to `opts.url`.

### 2. `pack` and `unpack` tarballs locally

##### `baltar.unpack(opts)`

- `opts`: {Object|string} Options for unpacking tarball.
- `opts.path`: Directory or file to unpack to
- `returns`: {Stream} Gunzip and untar pipechain to `opts.path`.
<commit_msg>[doc] Update formatting.<commit_after># baltar
A few small utilities for working with tarballs and http. Because you need tarballs over HTTP like:

![](https://media.giphy.com/media/52HjuHsfVO69q/giphy.gif)

## Usage

### Fetch & send tarballs over the network

##### `baltar.pull(opts, callback)`

Makes a request to `opts.url` and unpacks it to `opts.path`. `baltar.pull` is the only method which accepts a callback so that it pass back all of the entries from the `tar.Extract` stream. The end of pipechain is also returned for future stream operations (if desired).

- `opts.url`: {string} Location of the receiver.
- `opts.headers`: {Object} HTTP headers to send.
- `opts.method`: {string} HTTP Method to send.
- `opts.path`: {string} Directory or file to unpack to.
- `opts.tarball`: {string} **Optional** Path to save tarball to.
- `returns`: {tar.Extract} Extraction stream for the pulled tarball.

##### `baltar.push(opts)`

- `opts.path`: Directory or file to pack
- `opts.ignoreFiles`: Extra ignore files to parse
- `opts.url`: {string} Location of the receiver.
- `opts.headers`: {Object} HTTP headers to send.
- `opts.method`: {string} HTTP Method to send.
- `returns`: {Stream} HTTP request stream to `opts.url`.

### Pack and unpack tarballs locally

##### `baltar.unpack(opts)`

- `opts`: {Object|string} Options for unpacking tarball.
- `opts.path`: Directory or file to unpack to
- `returns`: {Stream} Gunzip and untar pipechain to `opts.path`.

{'fraction_non_alphanumeric': 0.1378839590443686, 'fraction_numerical': 0.0034129692832764505, 'mean_word_length': 4.2715827338129495, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2154089', 'n_tokens_mistral': 985, 'n_tokens_neox': 909, 'n_words': 367}