<commit_before>- Multi-threading in feature extraction
- New features
  - Region-of-interest, centered rectangle/ellipse in the image
  - Increase median filter size (color)
  - Experiment blur/sharpen filtering (shape)
  - Apply threshold on returned images distance, so that
    actual #returned <= #requested
- Use msgpack or protobuf for tcp protocol
- Cleanup core classes (Descriptors, Index)<commit_msg>Updated TODO list<commit_after>- Multi-threading in feature extraction
- New features
  - Region-of-interest, centered rectangle/ellipse in the image
  - Increase median filter size (color)
  - Experiment blur/sharpen filtering (shape)
  - Apply threshold on returned images distance, so that
    actual #returned <= #requested
- Use msgpack or protobuf for tcp protocol
- Find a better logging library
- Cleanup core classes (Descriptors, Index)
{'fraction_non_alphanumeric': 0.0689252336448598, 'fraction_numerical': 0.0, 'mean_word_length': 4.225609756097561, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4051578', 'n_tokens_mistral': 230, 'n_tokens_neox': 220, 'n_words': 96}