<commit_before>//                           -*- mode: doc; outline-regexp: "==\(=*\) ." -*-
ltsmin(5)
=========
:Author:  <NAME>
:Email:   <<EMAIL>>
:Date:    2008-10-22

== NAME
ltsmin - file formats used by the ltsmin tools

== SYNOPSIS
File layout and compression of ltsmin files.

== DESCRIPTION
The file formats supported by the ltsmin tools are extensions
of the .dir LTS format used in the mCRL toolset. The extensions are:

*pattern mode*::
An LTS in _DIR_ format for W workers uses 3W^2^\+2 data stream written to one file each. The first generation
tools store these file in a directory which must have the extension .dir. Pattern mode
allows a more general mapping of stream to file name. To write in pattern mode you
specify a filename containing one occurrence of ++%s++. For example, a backwards compatible 
layout for ++lts.dir++ would be ++lts.dir/%s++. Another example, would be prefixing every files with
++pre++. This would be specified as ++pre%s++.
Any LTS file name containing ++%s++ will be assumed to be a pattern mode archive.

*archive mode*::
In archive mode all data streams files are written to a single archive file.
This single archive file is a home-grown archive file format called
Generic Archive Format. Unlike ZIP, this archive format supports
interleaved writing of multiple data streams by multiple processes.
We suggest using the extension +.gcf+.
Any LTS file name not containing an occurrence of ++%s++ will be assumed to be a GCF archive.

*compression*::
All tools can apply compression to an LTS file. Compression is enabled by default. It can be switched off
by giving the command line argument ++-plain++. Similar to a ZIP archive, compression is applied to separate
data streams. These fall into two categories: labels and states. The labels are compressed using gzip.
The states are compressed with difference encoding followed by gzip.
The tools will detect if the input file is comressed or not.<commit_msg>Fixed typos<commit_after>//                           -*- mode: doc; outline-regexp: "==\(=*\) ." -*-
ltsmin(5)
=========
:Author:  <NAME>
:Email:   <<EMAIL>>
:Date:    2008-10-22

== NAME
ltsmin - file formats used by the ltsmin tools

== SYNOPSIS
File layout and compression of ltsmin files.

== DESCRIPTION
The file formats supported by the ltsmin tools are extensions
of the .dir LTS format used in the mCRL toolset. The extensions are:

*pattern mode*::
An LTS in _DIR_ format for W workers uses 3W^2^\+2 data stream written to one file each. The first generation
tools store these file in a directory which must have the extension .dir. Pattern mode
allows a more general mapping of stream to file name. To write in pattern mode you
specify a filename containing one occurrence of ++%s++. For example, a backwards compatible 
layout for ++lts.dir++ would be ++lts.dir/%s++. Another example, would be prefixing every files with
++pre++. This would be specified as ++pre%s++.
Any LTS file name containing ++%s++ will be assumed to be a pattern mode archive.

*archive mode*::
In archive mode all data stream files are written to a single archive file.
This single archive file is a home-grown archive file format called
Generic Archive Format. Unlike ZIP, this archive format supports
interleaved writing of multiple data streams by multiple processes.
We suggest using the extension +.gcf+.
Any LTS file name not containing an occurrence of ++%s++ will be assumed to be a GCF archive.

*compression*::
All tools can apply compression to an LTS file. Compression is enabled by default. It can be switched off
by giving the command line argument ++-plain++. Similar to a ZIP archive, compression is applied to separate
data streams. These fall into two categories: labels and states. The labels are compressed using gzip.
The states are compressed with difference encoding followed by gzip.
The tools will detect if the input file is compressed or not.
{'fraction_non_alphanumeric': 0.07668711656441718, 'fraction_numerical': 0.006134969325153374, 'mean_word_length': 4.0038363171355495, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '601295', 'n_tokens_mistral': 1052, 'n_tokens_neox': 1014, 'n_words': 596}