<commit_before>Prefetch Benchmarking
=====
<commit_msg>edited readme<commit_after>Prefetch Benchmarking
=====

**Usage:**

To make the binaries and prepare for testing just run:

   ./make_and_copy.sh

This is will compile the necessary files and copy the binaries to **this** directory for testing.

To configure the testing, you will have to edit the prefetch_benchmark.py file. No other files should need to be edited for basic configuration.

To run the test:

   ./prefetch_benchmark.py

Expects the input file to be called 'check0' to be a simple list of reads separated by newlines. You can always edit the variable INPUT_READ_FILE to change the input file name.

The output files will be in **this** directory and will be called 'constant_prefetch.dat' assuming you do not make any changes to the configuration. You can always go into prefetch_benchmark.py and edit the variable OUTPUT_BENCHMARK_FILE. You s
{'fraction_non_alphanumeric': 0.05683060109289618, 'fraction_numerical': 0.001092896174863388, 'mean_word_length': 4.6894409937888195, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3368246', 'n_tokens_mistral': 251, 'n_tokens_neox': 241, 'n_words': 128}