<commit_before>brotex
======

Attempt to implement the core functionality of vortex using bro-code<commit_msg>Add some example and documentation<commit_after>brotex
======

Attempt to implement the core functionality of vortex using bro-code

Tested with Bro 2.3 on SecurityOnion

Look at exec-bro.sh for an example of how to exec bro with these scripts.

```
ckane@cyberdef-so:~/bro-stream-test$ sudo bro -i eth1 -b -e 'redef Conn::extract_folder = "/nsm/sensor_data/cyberdef-so-eth1/brotex/"; redef Conn::stream_ext = ".smtp"' ./local.bro 2> stderr.log
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39844-192.168.6.1:25_CaP9x4U9dqufXRWJc_orig.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39844-192.168.6.1:25_CaP9x4U9dqufXRWJc_resp.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39845-192.168.6.1:25_CbQGo41m9TzqvO7ARf_orig.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39845-192.168.6.1:25_CbQGo41m9TzqvO7ARf_resp.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39846-192.168.6.1:25_CENHWptqKBXh8F36b_orig.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39846-192.168.6.1:25_CENHWptqKBXh8F36b_resp.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39847-192.168.6.1:25_C5QHMRcv9ZBWHH3ak_orig.smtp
/nsm/sensor_data/cyberdef-so-eth1/brotex/stream_192.168.6.2:39847-192.168.6.1:25_C5QHMRcv9ZBWHH3ak_resp.smtp
```
{'fraction_non_alphanumeric': 0.14175977653631286, 'fraction_numerical': 0.15572625698324022, 'mean_word_length': 5.368888888888889, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 16, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2159167', 'n_tokens_mistral': 816, 'n_tokens_neox': 685, 'n_words': 69}