<commit_before><!-- This form is for bug reports and feature requests only.

If you're looking for help check [Discord](https://discord.gg/sia) and the Sia [website](sia.tech).

Please make sure the bug report doesn't already exist in the GitHub issues.
-->

# Is this a BUG REPORT or FEATURE REQUEST?

**Stack Trace or error message**

**Expected Behavior**
<commit_msg>implementing Luke's comments<commit_after><!-- If this is a feature request, please put 'Feature Request:' in the issue title. -->

# BUG REPORT

**Stack Trace or error message**

**Expected Behavior**

{'fraction_non_alphanumeric': 0.11169284467713787, 'fraction_numerical': 0.0, 'mean_word_length': 4.415094339622642, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2850696', 'n_tokens_mistral': 170, 'n_tokens_neox': 162, 'n_words': 76}