<commit_before>Running the Crawler
===================

Once you have a [configuration](configuration.md), you can start running your crawler.  The crawler exposes 4 subcommands, accessible through the main binary at `bin/crawler`.  These commands are:

*  `setup` - Prepare for crawling (used for creating database schemas, creating log directories, etc).
*  `crawl` - Execute a crawl session
*  `teardown` - Remove any artifacts (used for removing database schemas, tearing down database tables, etc).
*  `reset` - A combination of `setup` and `teardown`.  Used for resetting the workspace between crawler runs.

Example of using the crawl command to run the sample configuration:

```bash
vendor/bin/crawler crawl -c docs/sample.php
```<commit_msg>Update docs to note command syntax change<commit_after>Running the Crawler
===================

Once you have a [configuration](configuration.md), you can start running your crawler.  The crawler exposes 4 subcommands, accessible through the main binary at `bin/crawler`.  These commands are:

*  `setup` - Prepare for crawling (used for creating database schemas, creating log directories, etc).
*  `crawl` - Execute a crawl session
*  `teardown` - Remove any artifacts (used for removing database schemas, tearing down database tables, etc).
*  `reset` - A combination of `setup` and `teardown`.  Used for resetting the workspace between crawler runs.

Example of using the crawl command to run the sample configuration:

```bash
vendor/bin/crawler crawl docs/sample.php
```
{'fraction_non_alphanumeric': 0.10281597904387688, 'fraction_numerical': 0.0013097576948264572, 'mean_word_length': 4.809885931558935, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '713721', 'n_tokens_mistral': 426, 'n_tokens_neox': 394, 'n_words': 189}