<commit_before>Turbine - Message Queue
=======================

Turbine is a Kafka like message queue that is based on Redis.

                   Current last element of pipeline
                   ❙
                   ⬇
    _____________________
    |1|2|3|3|4|5|6|7|8|9|
    ⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺
     ⬆       ⬆
     ❙       ❙
     ❙       Pointer for consumer 1
     Pointer for consumer 2
<commit_msg>added Readme<commit_after>Turbine - Message Queue
=======================

Turbine is a Kafka like message queue that is based on Redis.

                   Current last element of pipeline
                   ❙
                   ⬇
    _____________________
    |1|2|3|3|4|5|6|7|8|9|
    ⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺⎺
     ↑       ↑
     ❙       ❙
     ❙       Pointer for consumer 1
     Pointer for consumer 2

{'fraction_non_alphanumeric': 0.16563658838071693, 'fraction_numerical': 0.029666254635352288, 'mean_word_length': 1.6644736842105263, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '549377', 'n_tokens_mistral': 373, 'n_tokens_neox': 358, 'n_words': 75}