<commit_before># Development Guide

## Building

There's some `package.json` scripts for the project, so compilation _should_ be easy. There's two targets right now,
native and JS. Native is mostly used for testing and not distributed any where. The JS target is what
is distributed on NPM and is interfaced by `meow`.

Currently, OCaml >= 4.03 is required.

Building requires some packages from [OPAM](https://opam.ocaml.org).
The following command will install them.
If you already have [Reason](https://reasonml.github.io) installed (for example, through NPM) you can omit it.

```
$ opam install jbuilder reason sedlex js_of_ocaml flow_parser
```

To build for native:

```
$ npm run build:native
```

The native target is great for testing because it compiles super fast! However, it does not run `refmt`
on the results so be wary of syntax errors. To run the native binary, use:

```<commit_msg>Explain installing the JS dependences<commit_after># Development Guide

## Building

There's some `package.json` scripts for the project, so compilation _should_ be easy. There's two targets right now,
native and JS. Native is mostly used for testing and not distributed any where. The JS target is what
is distributed on NPM and is interfaced by `meow`.

Currently, OCaml >= 4.03 is required.

Building requires some packages from [OPAM](https://opam.ocaml.org).
The following command will install them.
If you already have [Reason](https://reasonml.github.io) installed (for example, through NPM) you can omit it.

```
$ opam install jbuilder reason sedlex js_of_ocaml flow_parser
```

Then, in the root of the project, install JS dependencies with `yarn` or `npm install`.

To build for native:

```
$ npm run build:native
```

The native target is great for testing because it compiles super fast! However, it does not run `refmt`
on the results so be wary of syntax errors. To run the native binary, use:

```
{'fraction_non_alphanumeric': 0.08093994778067885, 'fraction_numerical': 0.003133159268929504, 'mean_word_length': 4.505747126436781, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2167933', 'n_tokens_mistral': 555, 'n_tokens_neox': 524, 'n_words': 273}