<commit_before>parsed
======

Simple parser builder written on Python. Inspired by arpeggio parser
(http://arpeggio.googlecode.com/) but it was not suitable for my needs
and taste so it was simplier to write own parser generator.<commit_msg>starting to document parsed<commit_after>parsed
======

Simple parser builder written on Python. Inspired by arpeggio parser
(http://arpeggio.googlecode.com/) but it was not suitable for my needs
and taste so it was simplier to write own parser generator.

Writing parsers
---------------

To create parser and use predefined rules one need to import parsed
package.

from parsed import *

Parser generator function accepts no parameters and is decorated by
@rule, it should return a rule, e.g.:

        @rule
        def abc(): return char('abc') > value

As expected result of calling a generator function is a parser. Parser
gets its name from generator function name. This is why lambdas
etc. are not used. To correctly parse iterable source one should wrap
it with the source() function.

        parse = abc()
        src = source('a')
        position, result = parse(src)

### Rules

#### Character matching rule

Matches character against single character, set of characters passed
in the string or predicate accepting character and returning True on
matching. Examples are:

* char('A') # matches against 'A', on match by default skips/ignores
        it

* char('\r\n') # matches against LF or CR, on match by default returns
        matched character

* against predicate

        import string
        def is_punct(c): return c in string.punctuation

        char(is_punct) # matches against predicate, on match by
        default returns the character
{'fraction_non_alphanumeric': 0.07439953134153486, 'fraction_numerical': 0.0, 'mean_word_length': 3.4248704663212437, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2288857', 'n_tokens_mistral': 438, 'n_tokens_neox': 426, 'n_words': 223}