<commit_before>#Assemblathon Codebase

This is a set of evaluation scripts developed by [<NAME>](https://github.com/dentearl/) and [<NAME>](https://github.com/benedictpaten/) to assess the [Assemblathon](http://assemblathon.org/) 1 dataset.

##Dependencies

* [jobTree](https://github.com/benedictpaten/jobTree)
* [sonLib](https://github.com/benedictpaten/sonLib)
* [cactus](https://github.com/benedictpaten/cactus)
* [assemblaLib](https://github.com/benedictpaten/assemblaLib)

##Running Tests
Download the data set [here](http://hgwdev.cse.ucsc.edu/~benedict/code/Assemblathon_1_files/data.zip), expand it and place the data/ directory in the same directory as this README. Then type

    make test

and off you go.
<commit_msg>changed README to suggest the smallest test first.<commit_after>#Assemblathon Codebase

This is a set of evaluation scripts developed by [<NAME>](https://github.com/dentearl/) and [<NAME>](https://github.com/benedictpaten/) to assess the [Assemblathon](http://assemblathon.org/) 1 dataset.

##Dependencies

* [jobTree](https://github.com/benedictpaten/jobTree)
* [sonLib](https://github.com/benedictpaten/sonLib)
* [cactus](https://github.com/benedictpaten/cactus)
* [assemblaLib](https://github.com/benedictpaten/assemblaLib)

##Running Tests
Download the data set [here](http://hgwdev.cse.ucsc.edu/~benedict/code/Assemblathon_1_files/data.zip), expand it and place the data/ directory in the same directory as this README. Then type

    make testLittle

and off you go. There are larger and longer running tests inside of the Makefile if you'd like to justify spending a couple hours doing something else.

{'fraction_non_alphanumeric': 0.13117754728492984, 'fraction_numerical': 0.0024405125076266015, 'mean_word_length': 5.188679245283019, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7598449', 'n_tokens_mistral': 546, 'n_tokens_neox': 527, 'n_words': 138}