<commit_before>NMEA0183 State Machine
======================

Character Groups
----------------

|Name    |Value          |
|--------|---------------|
|RESERVED|`\x0a\x0d!$*,^`|
|FUTURE  |`\x5c\x7e-\xff`|
|CR      |`\x0d`         |
|LF      |`\x0a`         |

Rules
-----

(All rules are in priority order.)

|Current state    |Input      |Next state       |
|-----------------|-----------|-----------------|
|*                |FUTURE     |RESET            |
|*                |`$`        |ADDRESS          |
|ADDRESS          |`A-Z0-9`   |ADDRESS          |
|ADDRESS          |`,`        |FIELD_DATA       |
|FIELD_DATA       |`,`        |FIELD_DATA       |
|FIELD_DATA       |`^`        |FIELD_DATA_HEX_HI|
|FIELD_DATA_HEX_HI|`0-9A-F`   |FIELD_DATA_HEX_LO|
|FIELD_DATA_HEX_LO|`0-9A-F`   |FIELD_DATA       |
|FIELD_DATA       |`*`        |CHECKSUM_HI      |
|CHECKSUM_HI      |`0-9A-F`   |CHECKSUM_LO      |
|CHECKSUM_LO      |`0-9A-F`   |EOS_CR           |
|EOS_CR           |CR         |EOS_LF           |
|EOS_LF           |LF         |ACCEPT           |
|*                |RESERVED   |RESET            |
|FIELD_DATA       |*          |FIELD_DATA       |
|*                |*          |RESET            |<commit_msg>Whitespace cleanup<commit_after>NMEA0183 State Machine
======================

Character Groups
----------------

|Name    |Value          |
|--------|---------------|
|RESERVED|`\x0a\x0d!$*,^`|
|FUTURE  |`\x5c\x7e-\xff`|
|CR      |`\x0d`         |
|LF      |`\x0a`         |

Rules
-----

(All rules are in priority order.)

|Current state    |Input      |Next state       |
|-----------------|-----------|-----------------|
|*                |FUTURE  |RESET            |
|*                |`$`     |ADDRESS          |
|ADDRESS          |`A-Z0-9`|ADDRESS          |
|ADDRESS          |`,`     |FIELD_DATA       |
|FIELD_DATA       |`,`     |FIELD_DATA       |
|FIELD_DATA       |`^`     |FIELD_DATA_HEX_HI|
|FIELD_DATA_HEX_HI|`0-9A-F`|FIELD_DATA_HEX_LO|
|FIELD_DATA_HEX_LO|`0-9A-F`|FIELD_DATA       |
|FIELD_DATA       |`*`     |CHECKSUM_HI      |
|CHECKSUM_HI      |`0-9A-F`|CHECKSUM_LO      |
|CHECKSUM_LO      |`0-9A-F`|EOS_CR           |
|EOS_CR           |CR      |EOS_LF           |
|EOS_LF           |LF      |ACCEPT           |
|*                |RESERVED|RESET            |
|FIELD_DATA       |*       |FIELD_DATA       |
|*                |*       |RESET            |
{'fraction_non_alphanumeric': 0.22444722569879016, 'fraction_numerical': 0.016687526074259492, 'mean_word_length': 1.1072056239015817, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '848548', 'n_tokens_mistral': 892, 'n_tokens_neox': 844, 'n_words': 117}