<commit_before>To run all tests, you can execute the general run-tests.py program.

You need to go into its "grandfather" directory, ../.. (i.e. Deviser/deviser/),
and invoke this command:

    python -m generator.tests.run-tests

If you wish to execute a specific subset of tests, e.g. test_cpp_code/run_cpp_tests.py,
you can comment out all other test subdirs in run-tests.py, then run that. This is not
ideal, but is a result of the new relative import directives which have removed the
need for horrible lines such as

    sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/../')

etc.

This is all part of our move to using pytest to run our tests.

To run tests in this directory, you may need to copy

    Deviser/deviser/generator/pytest_files/test_xml_files/

into this current directory (Deviser/deviser/generator/tests/)


<commit_msg>Added caveat that old tests don't work with new imports under Python 2.<commit_after>To run all tests, you can execute the general run-tests.py program.

You need to go into its "grandfather" directory, ../.. (i.e. Deviser/deviser/),
and invoke this command:

    python -m generator.tests.run-tests

If you wish to execute a specific subset of tests, e.g. test_cpp_code/run_cpp_tests.py,
you can comment out all other test subdirs in run-tests.py, then run that. This is not
ideal, but is a result of the new relative import directives which have removed the
need for horrible lines such as

    sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/../')

etc.

This is all part of our move to using pytest to run our tests.

To run tests in this directory, you may need to copy

    Deviser/deviser/generator/pytest_files/test_xml_files/

into this current directory (Deviser/deviser/generator/tests/)

Unfortunately, these old tests don't seem to run right now with Python 2. Sorry.

{'fraction_non_alphanumeric': 0.07982740021574973, 'fraction_numerical': 0.0010787486515641855, 'mean_word_length': 3.7686375321336762, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4302009', 'n_tokens_mistral': 564, 'n_tokens_neox': 549, 'n_words': 246}