<commit_before>viscimja
========

A portable SYCPOL-SCIM in Java.

Features
========

* Full compatibility with Standard SYCPOL
* Support for filesystem access
* Advanced debug mode
* Verbose error messages

Building
========

VISCIM-JA requires Java 7.

Following should work:
```
$ mkdir classes
$ make
```

Or if you don't can or want use Makefiles:
```
$ java -classpath classes -sourcepath src src/sycpol/Sycpol.java -d classes
```

Usage
=====

See manpage for full documentation.<commit_msg>better english and more features<commit_after>viscimja
========

A portable SYCPOL-SCIM in Java.

Features
========

* Full compatibility with Standard SYCPOL
* MODULE statements and other essential features from RS-SYCPOL
* Support for filesystem access
* Advanced debug mode
* Verbose error messages

Building
========

VISCIM-JA requires Java 7.

Following should work:
```
$ mkdir classes
$ make
```

Or if you aren't able or would rather not use Makefiles:
```
$ java -classpath classes -sourcepath src src/sycpol/Sycpol.java -d classes
```

Usage
=====

See manpage for full documentation.
{'fraction_non_alphanumeric': 0.12076852698993595, 'fraction_numerical': 0.0018298261665141812, 'mean_word_length': 4.553299492385787, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1904368', 'n_tokens_mistral': 347, 'n_tokens_neox': 322, 'n_words': 125}