<commit_before>API incompatibility
-------------------

V0.1.*: * First formal version of API

V0.2.*: * worker_start takes a 'worker_t' as an additional parameter
	(This enables to perform a wait-on-event instead of a busy waiting).
        * worker_spawn doesn't take into a 'worker_t' anymore.
	* worker_init takes an int as numWorker instead of an unsigned int.
	* RBTREE pop method has been renamed into _pop_at (as it takes a key parameter).
        * SNAPSHOT_DEF has been removed. use SNAPSHOT_SPSC_DEF
        * array's _set_at2 has been replaced by array's _get_at.
        The interface is similar to dict _get_at, making the API cleaner and more generic.
            _set_at2(a,i,x) ==> set(*_get_at(a,i),x)
        * DICT_SET:
          - Rename method _set_at as _push
        Since there is no key, it is a push method, not a _set_at method.
          - Rename DICT_SET_DEF2 as DICT_SET_DEF
        since it only requires one argument.
          - The method _ref returns a pointer to the data,
        instead of a pointer to a single tuple like other DICT_SET interface.
          - The format of the output to character string as changed. 
        * ALGO:
          - Rename _contains as _contain_p
        since it is a predicate '_p' and it shall be in infinitive form.
          - Rename _map as _for_each (Standard name).
        * M_PRINT_ARG & M_FPRINT_ARG don't put a final semicolon at the end
        of their expansion anymore (cleaner and no longer needed with the
        new available tools).
        * For the TUPLE and the VARIANT
        change name of _get_ ## field into _get_at_ ## fields
        Since there is a name collision with other methods
        if a field is named 'str'.
        * Rename method _remove into _erase for dictionary & B+TREE (adding a compatibility layer) for API uniformization.
        The provided _remove doesn't use an iterator but a 'key', incompatible with the one of array or list.
        But is like the one provided by the priority queue.
        
V0.3.*: * Remove B+TREE _remove method as it has been deprecated (use _erase instead)
        * Remove DICT _remove method as it has been deprecated (use _erase instead)
        * Make _it_to_p method of B+TREE / RB-TREE as deprecated (uses _it_until_p instead ).
        * Deprecated buffer options BUFFER_BLOCKING* and BUFFER_UNBLOCKING*
        as the _push_blocking and _pop_blocking functions can do the same and are more versatil.
        <commit_msg>Update new API change<commit_after>API incompatibility
-------------------

V0.1.*: * First formal version of API

V0.2.*: * worker_start takes a 'worker_t' as an additional parameter
	(This enables to perform a wait-on-event instead of a busy waiting).
        * worker_spawn doesn't take into a 'worker_t' anymore.
	* worker_init takes an int as numWorker instead of an unsigned int.
	* RBTREE pop method has been renamed into _pop_at (as it takes a key parameter).
        * SNAPSHOT_DEF has been removed. use SNAPSHOT_SPSC_DEF
        * array's _set_at2 has been replaced by array's _get_at.
        The interface is similar to dict _get_at, making the API cleaner and more generic.
            _set_at2(a,i,x) ==> set(*_get_at(a,i),x)
        * DICT_SET:
          - Rename method _set_at as _push
        Since there is no key, it is a push method, not a _set_at method.
          - Rename DICT_SET_DEF2 as DICT_SET_DEF
        since it only requires one argument.
          - The method _ref returns a pointer to the data,
        instead of a pointer to a single tuple like other DICT_SET interface.
          - The format of the output to character string as changed. 
        * ALGO:
          - Rename _contains as _contain_p
        since it is a predicate '_p' and it shall be in infinitive form.
          - Rename _map as _for_each (Standard name).
        * M_PRINT_ARG & M_FPRINT_ARG don't put a final semicolon at the end
        of their expansion anymore (cleaner and no longer needed with the
        new available tools).
        * For the TUPLE and the VARIANT
        change name of _get_ ## field into _get_at_ ## fields
        Since there is a name collision with other methods
        if a field is named 'str'.
        * Rename method _remove into _erase for dictionary & B+TREE (adding a compatibility layer) for API uniformization.
        The provided _remove doesn't use an iterator but a 'key', incompatible with the one of array or list.
        But is like the one provided by the priority queue.
        
V0.3.*: * Remove B+TREE _remove method as it has been deprecated (use _erase instead)
        * Remove DICT _remove method as it has been deprecated (use _erase instead)
        * Make _it_to_p method of B+TREE / RB-TREE as deprecated (uses _it_until_p instead ).
        * Deprecated buffer options BUFFER_BLOCKING* and BUFFER_UNBLOCKING*
        as the _push_blocking and _pop_blocking functions can do the same and are more versatil.
	* Iteration over the items of a dictionnary/tree returns const iterators for access through
	OPLIST (impact: algorithm & M_EACH) as the IT_REF operator is no longer exported.
	The _it_ref method is still defined and can be used, so you can still modify the values
	but it won't be automatically used anymore (as the keys CANNOT be modified).
	(Previously it generates incorrect algorithm when used with m-dict).
{'fraction_non_alphanumeric': 0.05623836126629423, 'fraction_numerical': 0.0033519553072625698, 'mean_word_length': 2.656228727025187, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7264667', 'n_tokens_mistral': 1517, 'n_tokens_neox': 1500, 'n_words': 722}