<commit_before># ppbert

A simple command-line utility to pretty print structures encoded using
Erlang's [External Term Format](http://erlang.org/doc/apps/erts/erl_ext_dist.html).
The input is read from *stdin* and written to *stdout*, making ppbert
a good candidate for shell pipelines.

At the moment, ppbert supports only a subset of the field types of the
External Term Format:

- integers;
- big integers;
- floats;
- atoms (latin-1 and UTF-8);
- binaries;
- tuples;
- lists.

## Usage

```sh
$ ppbert <mini_dict.bert
[
  {host, "localhost"},
  {port, 80},
  {
    headers,
    [
      {
        <<"X-Real-Ip">>,
        {127, 0, 0, 1}
      },
      {<<"Keep-alive">>, true}
    ]
  }
]

$ printf "\x83\x77\x04atom" | ppbert
atom
```

## Performance

Ppbert is written in Rust and offers an appreciable performance gain
over using Erlang's `erlang:binary_to_term/1` and `io:format/2`.

```sh
$ cat erl_ppbert
#!/usr/bin/env escript

main(Args) ->
    lists:foreach(fun (Filename) ->
        {ok, Binary} = file:read_file(Filename),
        io:format("~p~n", [binary_to_term(Binary)])
    end, Args).

$ du large.bert
96M     large.bert

$ time ./erl_ppbert large.bert >/dev/null

real    1m10.968s
user    0m49.644s
sys     0m13.628s

$ time ppbert <large.bert >/dev/null

real    0m19.670s
user    0m19.092s
sys     0m0.432s
```

## Future work

- Add flags to control the pretty printing (e.g., indentation width,
  number of basic values on a single line, etc.).
- Add a [jq](https://stedolan.github.io/jq/)-like query language.
- Write a man page.<commit_msg>Update timing numbers after optimization<commit_after># ppbert

A simple command-line utility to pretty print structures encoded using
Erlang's [External Term Format](http://erlang.org/doc/apps/erts/erl_ext_dist.html).
The input is read from *stdin* and written to *stdout*, making ppbert
a good candidate for shell pipelines.

At the moment, ppbert supports only a subset of the field types of the
External Term Format:

- integers;
- big integers;
- floats;
- atoms (latin-1 and UTF-8);
- binaries;
- tuples;
- lists.

## Usage

```sh
$ ppbert <mini_dict.bert
[
  {host, "localhost"},
  {port, 80},
  {
    headers,
    [
      {
        <<"X-Real-Ip">>,
        {127, 0, 0, 1}
      },
      {<<"Keep-alive">>, true}
    ]
  }
]

$ printf "\x83\x77\x04atom" | ppbert
atom
```

## Performance

Ppbert is written in Rust and offers an appreciable performance gain
over using Erlang's `erlang:binary_to_term/1` and `io:format/2`.

```sh
$ cat erl_ppbert
#!/usr/bin/env escript

main(Args) ->
    lists:foreach(fun (Filename) ->
        {ok, Binary} = file:read_file(Filename),
        io:format("~p~n", [binary_to_term(Binary)])
    end, Args).

$ du large.bert
96M     large.bert

$ time ./erl_ppbert large.bert >/dev/null

real    1m10.968s
user    0m49.644s
sys     0m13.628s

$ time ppbert <large.bert >/dev/null

real    0m6.902s
user    0m6.116s
sys     0m0.452s
```

## Future work

- Add flags to control the pretty printing (e.g., indentation width,
  number of basic values on a single line, etc.).
- Add a [jq](https://stedolan.github.io/jq/)-like query language.
- Write a man page.
{'fraction_non_alphanumeric': 0.13594688586784698, 'fraction_numerical': 0.03414479924122668, 'mean_word_length': 3.020330368487929, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 19, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1578627', 'n_tokens_mistral': 1224, 'n_tokens_neox': 1133, 'n_words': 341}