<commit_before>= Reference

[partintro]
--
This part of the documentation explains the core functionality of {eh} starting with the configuration options and architecture and gradually explaining the various major features. We recommend going through the entire documentation even superficially when trying out {eh} for the first time, however those in a rush, can jump directly to the desired sections:

<<architecture>>:: overview of the {eh} architecture and how it maps on top of {mr}

<<configuration>>:: explore the various configuration switches in {eh}
<commit_msg>add section ids (plus float)<commit_after>[[reference]]
= Reference

[partintro]
--
This part of the documentation explains the core functionality of {eh} starting with the configuration options and architecture and gradually explaining the various major features. We recommend going through the entire documentation even superficially when trying out {eh} for the first time, however those in a rush, can jump directly to the desired sections:

<<arch>>:: overview of the {eh} architecture and how it maps on top of {mr}

<<configuration>>:: explore the various configuration switches in {eh}

{'fraction_non_alphanumeric': 0.06341045415595545, 'fraction_numerical': 0.0, 'mean_word_length': 5.561797752808989, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3006605', 'n_tokens_mistral': 260, 'n_tokens_neox': 253, 'n_words': 162}