<commit_before># coding: utf-8
"""
    regex
    ~~~~~

    This is a prototype for an implementation of regular expressions. The goal
    of this prototype is to develop a completely transparent implementation,
    that can be better reasoned about and used in a parser.

    Note that as of now "regular expressions" actually means *regular*, so
    non-greedy repetitions (``*?``, ``+?``), positive/negative
    lookahead/-behind assertions are not supported with the current
    implementation. On the other hand this allows for a very efficient
    implementation.

    Currently regular expressions can be compiled to NFAs and DFAs (graph and
    table driven).

    :copyright: 2012 by <NAME>
    :license: BSD, see LICENSE.rst
"""
<commit_msg>Add warning about real-world usage<commit_after># coding: utf-8
"""
    regex
    ~~~~~

    This is a prototype for an implementation of regular expressions. The goal
    of this prototype is to develop a completely transparent implementation,
    that can be better reasoned about and used in a parser. This is not meant
    to be an implementation that will see real-world usage, in fact the
    performance on most - if not all - python implementations will make that
    prohibitive.

    Note that as of now "regular expressions" actually means *regular*, so
    non-greedy repetitions (``*?``, ``+?``), positive/negative
    lookahead/-behind assertions are not supported with the current
    implementation. On the other hand this allows for a very efficient
    implementation.

    Currently regular expressions can be compiled to NFAs and DFAs (graph and
    table driven).

    :copyright: 2012 by <NAME>
    :license: BSD, see LICENSE.rst
"""

{'fraction_non_alphanumeric': 0.07147041593438781, 'fraction_numerical': 0.005858230814294083, 'mean_word_length': 3.3794871794871795, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2857910', 'n_tokens_mistral': 445, 'n_tokens_neox': 408, 'n_words': 218}