<commit_before>* .mobi processing
** TODO check bufferpack dependency
** TODO huff/cdic compression
** TODO extract images
** TODO cope with overlap multibytes
* HTML transformations
** TODO filepos offset links
** TODO mbp:pagebreak tags
** TODO font sizes
** DONE A-with-ring entries
   CLOSED: [2014-01-01 Wed 19:29]
Example: in LastColony get a sequence like
6e 'n'
c3a2   U+00E2
c280   U+0080
c299   U+0099
73 's'
at a place where you'd expect a single quote mark (i.e. U+0039 or U+2019).
** TODO validate HTML to find other problems
* Testing framework
** DONE better way to insert/display generated book
   CLOSED: [2013-02-24 Sun 11:30]
** TODO way to inject a specific book to run against
** TODO run over entire set of available eBooks
** TODO run jslint automatically<commit_msg>Add ToDo entries for plugins<commit_after>* .mobi processing
** TODO check bufferpack dependency
** TODO huff/cdic compression
** TODO extract images
** TODO cope with overlap multibytes
* HTML transformations
** TODO filepos offset links
** TODO mbp:pagebreak tags
** TODO font sizes
** DONE A-with-ring entries
   CLOSED: [2014-01-01 Wed 19:29]
Example: in LastColony get a sequence like
6e 'n'
c3a2   U+00E2
c280   U+0080
c299   U+0099
73 's'
at a place where you'd expect a single quote mark (i.e. U+0039 or U+2019).
** TODO validate HTML to find other problems
* Testing framework
** DONE better way to insert/display generated book
   CLOSED: [2013-02-24 Sun 11:30]
** TODO way to inject a specific book to run against
** TODO run over entire set of available eBooks
** TODO run jslint automatically
* Plugins
** Chrome
** Firefox
** Safari
** IE?
{'fraction_non_alphanumeric': 0.08642726719415703, 'fraction_numerical': 0.06573341448569689, 'mean_word_length': 4.286173633440514, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1100051', 'n_tokens_mistral': 583, 'n_tokens_neox': 504, 'n_words': 228}