<commit_before>kanzi
=====


This project offers Java & Go code for manipulation and compression of data and images.
The goal is to provide clean APIs and really fast implementation.
It includes lossless compression codecs (Huffman, Range, LZ4, Snappy, PAQ), color model transforms, resampling, wavelet, DCT, Hadamard transform, bit stream manipulation, Burrows-Wheeler (BWT), Distance Coding and Move-To-Front transform, run length coding, etc ...
It also provides video filters such as fast Gaussian filter, Sobel filter and constant time bilateral filter.


For more details, check https://github.com/flanglet/kanzi/wiki or go to https://code.google.com/p/kanzi/<commit_msg>Remove DC from dexcription.<commit_after>kanzi
=====


This project offers Java & Go code for manipulation and compression of data and images.
The goal is to provide clean APIs and really fast implementation.
It includes lossless compression codecs (Huffman, Range, LZ4, Snappy, PAQ), color model transforms, resampling, wavelet, DCT, Hadamard transform, bit stream manipulation, Burrows-Wheeler (BWT) and Move-To-Front transform, Run Length coding, etc ...
It also provides video filters such as fast Gaussian filter, Sobel filter and constant time bilateral filter.


For more details, check https://github.com/flanglet/kanzi/wiki or go to https://code.google.com/p/kanzi/
{'fraction_non_alphanumeric': 0.07846039970392302, 'fraction_numerical': 0.0014803849000740192, 'mean_word_length': 4.955947136563877, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '434159', 'n_tokens_mistral': 372, 'n_tokens_neox': 351, 'n_words': 168}