<commit_before>Regression test
===============

The regression test executes a series of test cases which fully describe
OpenFAST and its module's capabilities. Each locally computed result is compared
to a static set of baseline results. To account for machine and compiler
differences, the regression test attempts to match the current machine and
compiler type to the appropriate solution set from these combinations

- macOS with GNU compiler (default)
- Red Hat Enterprise Linux with Intel compiler
- Windows with Intel compiler

The automated regression test runs
`CTest <https://cmake.org/Wiki/CMake/Testing_With_CTest>`__ and can be executed
by running either of the commands ``make test`` or ``ctest`` from the build
directory. If the entire OpenFAST package is to be built, CMake will configure
CTest to find the new binary at ``openfast/build/glue-codes/fast/openfast``.
However, if the intention is to build only the test suite, the OpenFAST binary
should be specified in the CMake configuration under the ``CTEST_OPENFAST_EXECUTABLE``
flag. There is also a corresponding ``CTEST_[MODULE]_NAME`` flag for each module
included in the regression test.

The regression test can be executed manually with the included driver
``reg_tests/manualRegressionTest.py``.

In both modes of execution a subdirectory is created in the build directory
called ``reg_tests`` where all of the input files for the test cases are copied
and all of the locally generated outputs are stored.

Ultimately, both CTest and the manual execution script call a series of Python
scripts and libraries in ``reg_tests`` and ``reg_tests/lib``. One such script is
``lib/pass_fail.py`` which reads the output files and computes a norm on each 
channel reported. If the maximum norm is greater than a preset tolerance, that particular
test is reported as failed. The failure criteria is outlined in pseudocode below.<commit_msg>update the intro for more clarity<commit_after>Regression test
===============

The regression test executes a series of test cases which fully describe
OpenFAST and its module's capabilities. Each locally computed result is compared
to a static set of baseline results. To account for machine and compiler
differences, the regression test attempts to match the current machine and
compiler type to the appropriate solution set from these combinations

- macOS with GNU compiler (default)
- Red Hat Enterprise Linux with Intel compiler
- Windows with Intel compiler

The regression test system can be executed with CMake and CTest or manually with
an included Python driver. Both systems provide similar functionality with respect
to testing, but CTest integration provides access to multithreading, automation,
and test reporting via CDash. Both modes of execution require some configuration.

In both modes of execution a subdirectory is created in the build directory
called ``reg_tests`` where all of the input files for the test cases are copied
and all of the locally generated outputs are stored.

Ultimately, both CTest and the manual execution program call a series of Python
scripts and libraries in ``reg_tests`` and ``reg_tests/lib``. One such script is
``lib/pass_fail.py`` which reads the output files and computes a norm on each 
channel reported. If the maximum norm is greater than a preset tolerance, that particular
test is reported as failed. The failure criteria is outlined in pseudocode below.
{'fraction_non_alphanumeric': 0.04793919906460099, 'fraction_numerical': 0.0, 'mean_word_length': 5.067375886524823, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '814816', 'n_tokens_mistral': 800, 'n_tokens_neox': 773, 'n_words': 496}