<commit_before># About Kollos

## What is Kollos?

Kollos is the next phase of Marpa.  Today, Marpa's main implementation consist of these pieces:

+ a C library, called Libmarpa, which implements the core parse engine.  It's very low-level --
for example, there are no strings.  Rules, symbols, error messages, etc. are all integers.

+ a Perl/XS wrapper, also in C, which provides the necessary upper layer for Libmarpa.

+ pure Perl logic, which combined with the previous two becomes Marpa::R2, a Perl module

Kollos will replace the Perl/XS wrapper with Lua.
In the process Lua will become the language for Marpa's semantics --
that is, you'll be able to specify custom logic in Lua.
Lua is extremely minimal and lightweight, and is perfect for this purpose.
<commit_msg>Clarify the role of the Perl/XS<commit_after># About Kollos

## What is Kollos?

Kollos is the next phase of Marpa.  Today, Marpa's main implementation consist of these pieces:

+ a C library, called Libmarpa, which implements the core parse engine.  It's very low-level --
for example, there are no strings.  Rules, symbols, error messages, etc. are all integers.

+ a Perl/XS wrapper, also in C, which provides the necessary upper layer for Libmarpa and
contains most of the SLIF logic.

+ pure Perl logic, which combined with the previous two becomes Marpa::R2, a Perl module

Kollos will replace the Perl/XS wrapper with Lua.
In the process Lua will become the language for Marpa's semantics --
that is, you'll be able to specify custom logic in Lua.
Lua is extremely minimal and lightweight, and is perfect for this purpose.

{'fraction_non_alphanumeric': 0.05790784557907846, 'fraction_numerical': 0.0012453300124533001, 'mean_word_length': 4.117834394904459, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5742637', 'n_tokens_mistral': 469, 'n_tokens_neox': 438, 'n_words': 250}