<commit_before># Lit Benchmarks

Benchmarks for evaluating the performance of
[lit-html](https://github.com/Polymer/lit-html), along with alternative
implementations.

## Contents

- [Setup](#setup)
- [Flags](#flags)
- [Automatic mode](#automatic-mode)
- [Manual mode](#manual-mode)
- [Output](#output)
- [Adding benchmarks](#adding-benchmarks)
- [Example commands](#example-commands)

### Setup

```sh
git clone <EMAIL>:PolymerLabs/lit-benchmarks.git
cd lit-benchmarks
npm install
npm run benchmarks
```

### Flags

Flag                      | Default     | Description
------------------------- | ----------- | --------------------------------
`--help`                  | `false`     | Show documentation
`--host`                  | `127.0.0.1` | Which host to run on
`--port`                  | `0`         | Which port to run on (`0` for random free)
`--benchmark` / `-b`      | `*`         | Which benchmarks to run (`*` for all)
`--implementation` / `-i` | `lit-html`  | Which implementations to run (`*` for all)
`--trials` / `-t`         | `10`        | How many times to run each benchmark
`--manual` / `-m`         | `false`     | Don't run automatically, just show URLs and collect results

### Automatic mode

The default mode automatically launches Chrome with the selected
benchmarks/implementations (all lit-html benchmarks by default), runs the
benchmarks, reports the results back to the server, and prints them to the
terminal.

### Manual mode

If the `--manual` or `-m` flag is set, then no benchmarks will run
automatically. Instead, URLs for all selected benchmarks/implementations will be
printed on the terminal, and the server will listen indefinitely. Visiting any
of these URLs in any browser (or any other valid benchmark URL on the host) will
run the benchmark, report the results back to the server, and print them to the
terminal.

### Output

### Adding benchmarks

To add a new benchmark or implementation, add a directory within the
`benchmarks/` directory following the layout:

```
benchmarks/
└── <implementation>/
   └── <benchmark>/
       ├── index.html
       └── index.js
```

Each implementation directory should have its own `package.json` to isolate
dependencies between implementations. Running `npm install` from the top-level
of the repo will automatically run `npm install` within each implementation
directory.

Benchmark `.js` files should `import {registerBenchmark} from
'../../../client/lib/index.js';` and call `registerBenchmark(<fn>);` once, where
`<fn>` is the function that implements the benchmark. The registered benchmark
will begin running after a `setTimeout`. Timing will begin immediately before
calling `<fn>`. If `<fn>` returns a promise then timing will end after awaiting<commit_msg>Remove unused Output section from README<commit_after># Lit Benchmarks

Benchmarks for evaluating the performance of
[lit-html](https://github.com/Polymer/lit-html), along with alternative
implementations.

## Contents

- [Setup](#setup)
- [Flags](#flags)
- [Automatic mode](#automatic-mode)
- [Manual mode](#manual-mode)
- [Adding benchmarks](#adding-benchmarks)
- [Example commands](#example-commands)

### Setup

```sh
git clone <EMAIL>:PolymerLabs/lit-benchmarks.git
cd lit-benchmarks
npm install
npm run benchmarks
```

### Flags

Flag                      | Default     | Description
------------------------- | ----------- | --------------------------------
`--help`                  | `false`     | Show documentation
`--host`                  | `127.0.0.1` | Which host to run on
`--port`                  | `0`         | Which port to run on (`0` for random free)
`--benchmark` / `-b`      | `*`         | Which benchmarks to run (`*` for all)
`--implementation` / `-i` | `lit-html`  | Which implementations to run (`*` for all)
`--trials` / `-t`         | `10`        | How many times to run each benchmark
`--manual` / `-m`         | `false`     | Don't run automatically, just show URLs and collect results

### Automatic mode

The default mode automatically launches Chrome with the selected
benchmarks/implementations (all lit-html benchmarks by default), runs the
benchmarks, reports the results back to the server, and prints them to the
terminal.

### Manual mode

If the `--manual` or `-m` flag is set, then no benchmarks will run
automatically. Instead, URLs for all selected benchmarks/implementations will be
printed on the terminal, and the server will listen indefinitely. Visiting any
of these URLs in any browser (or any other valid benchmark URL on the host) will
run the benchmark, report the results back to the server, and print them to the
terminal.

### Adding benchmarks

To add a new benchmark or implementation, add a directory within the
`benchmarks/` directory following the layout:

```
benchmarks/
└── <implementation>/
   └── <benchmark>/
       ├── index.html
       └── index.js
```

Each implementation directory should have its own `package.json` to isolate
dependencies between implementations. Running `npm install` from the top-level
of the repo will automatically run `npm install` within each implementation
directory.

Benchmark `.js` files should `import {registerBenchmark} from
'../../../client/lib/index.js';` and call `registerBenchmark(<fn>);` once, where
`<fn>` is the function that implements the benchmark. The registered benchmark
will begin running after a `setTimeout`. Timing will begin immediately before
calling `<fn>`. If `<fn>` returns a promise then timing will end after awaiting
{'fraction_non_alphanumeric': 0.1354299218323941, 'fraction_numerical': 0.0036357025995273585, 'mean_word_length': 3.09375, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '158601', 'n_tokens_mistral': 1575, 'n_tokens_neox': 1451, 'n_words': 626}