<commit_before>========================================================================
       CONSOLE APPLICATION : UnitTest
========================================================================


This file contains a summary of what you will find in each of the files that
make up your UnitTest application.

UnitTest.dsp
    This file (the project file) contains information at the project level and
    is used to build a single project or subproject. Other users can share the
    project (.dsp) file, but they should export the makefiles locally.

UnitTest.cpp
    This is the main application source file.


/////////////////////////////////////////////////////////////////////////////
This program allows to see how each interface method can be used in openIL

NOTE: This program depends on cppunit, which can be found on Sourceforge.net. 

<commit_msg>small description added<commit_after>========================================================================
       CONSOLE APPLICATION : UnitTest
========================================================================


This file contains a summary of what you will find in each of the files that
make up your UnitTest application.

UnitTest.dsp
    This file (the project file) contains information at the project level and
    is used to build a single project or subproject. Other users can share the
    project (.dsp) file, but they should export the makefiles locally.

UnitTest.cpp
    This is the main application source file.


/////////////////////////////////////////////////////////////////////////////
This program allows to see how each interface method can be used in openIL

NOTE: This program depends on cppunit, which can be found on Sourceforge.net. 
      Currently this test several routines in ILTest.
 
      It runs the test on the shell, and report it's results.


{'fraction_non_alphanumeric': 0.26185344827586204, 'fraction_numerical': 0.0, 'mean_word_length': 2.9426751592356686, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6781630', 'n_tokens_mistral': 394, 'n_tokens_neox': 362, 'n_words': 212}