<commit_before># Unitex/GramLab Core Integration Tests [![Build Status](https://travis-ci.org/UnitexGramLab/unitex-core-tests.svg?branch=master)](https://travis-ci.org/UnitexGramLab/unitex-core-tests)

> [Unitex/GramLab][unitex] is an open source, cross-platform, multilingual, lexicon- and grammar-based corpus processing suite

This repository contains non-regression and memory error detection tests which are
run after a commit or before a nightly build release to check the [Unitex Core][unitex-core]
compilation. These tests should be run by the contributors before send a pull-request.

<p align="center">
  <img src="http://www-igm.univ-mlv.fr/~unitex/img/unitex-core-test.png" width="50%" height="50%" alt="Unitex Core Tests"/>
</p>

## Dependencies

1. Make sure you have already a `UnitexToolLogger` binary built including debug symbols with no optimization, 
   if not follow the instructions on the [Unitex Core][unitex-core] to compile it

1. `Bash` v4.2.0 is the minimum version required to run `unitex-core-test.sh`.<commit_msg>update README.md [ci skip]<commit_after># Unitex/GramLab Core Integration Tests [![Build Status](https://travis-ci.org/UnitexGramLab/unitex-core-tests.svg?branch=master)](https://travis-ci.org/UnitexGramLab/unitex-core-tests)

> [Unitex/GramLab][unitex] is an open source, cross-platform, multilingual, lexicon- and grammar-based corpus processing suite

This repository contains non-regression and memory error detection tests which are
run after a commit or before a nightly build release to check the [Unitex Core][unitex-core]
compilation. These tests should be run by the contributors before send a pull-request.

<p align="center">
  <img src="http://www-igm.univ-mlv.fr/~unitex/img/unitex-core-test.png" width="50%" height="50%" alt="Unitex Core Tests"/>
</p>

## Dependencies

1. To run a single `ulp` test, you don't need to use `unitex-core-test.sh`,
   you can simple type e.g. `UnitexToolLogger RunLog Compress/Compress_bin2.ulp`
   on the command line. However, if you are planning to run multiple tests on a
   regular basis, including memory error detection tests, the best approach is
   to continue reading and use `unitex-core-test.sh`

1. Make sure you have already a `UnitexToolLogger` binary built including debug symbols with no optimization, 
   if not follow the instructions on the [Unitex Core][unitex-core] to compile it

1. `Bash` v4.2.0 is the minimum version required to run `unitex-core-test.sh`.
{'fraction_non_alphanumeric': 0.10643464184540671, 'fraction_numerical': 0.008093889113719142, 'mean_word_length': 4.789227166276347, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1803686', 'n_tokens_mistral': 814, 'n_tokens_neox': 780, 'n_words': 273}