<commit_before>
== csv2f64

Converts CSV file to simple list of doubles

=== Usage:

----
$ ./csv2f64 nt_train2-10.csv nt_train2-10.dat
csv2f64: converted: rows: 10 cols: 60484
----

=== Debugging statements

Are found on branch 'sz'.<commit_msg>Note about purpose<commit_after>
== csv2f64

Converts CSV file to simple list of doubles

To support collaboration with ECP CODAR, SZ compression (Cappello and Di)

=== Usage:

----
$ ./csv2f64 nt_train2-10.csv nt_train2-10.dat
csv2f64: converted: rows: 10 cols: 60484
----

=== Debugging statements

Are found on branch 'sz'.
{'fraction_non_alphanumeric': 0.12412587412587413, 'fraction_numerical': 0.07692307692307693, 'mean_word_length': 3.8151260504201683, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7560036', 'n_tokens_mistral': 237, 'n_tokens_neox': 202, 'n_words': 66}