<commit_before># Sourcer

A parsing library for Python (version 3.6 and later).

```python
from sourcer import Grammar

g = Grammar(r'''
    start = "Hello" >> /[a-zA-Z]+/

    ignore Space = /[ \t]+/
    ignore Punctuation = "," | "." | "!" | "?"
''')

# Try it out:
result = g.parse('Hello, World!')
assert result == 'World'

result = g.parse('Hello Chief?!?!!')
assert result == 'Chief'
```

Notes:

* `>>` means "discard the the left hand side"
* `/.../` means "regular expression"


## Installation

```console
$ python3 -m pip install sourcer
```


## Why Use Sourcer?

Sometimes you have to parse things, and sometimes a regex won't cut it.

Things you might have to parse someday:

- log files
- business rules
- market data feeds
- equations
- queries
- user input
- domain specific languages
- obscure data formats
- legacy source code

So that's what this library is for. It's for when you have to take some text
and turn it into a tree of Python objects.


#### Aren't there a ton of parsing libraries for Python already?

Yes. But there's always so much ceremony involved. The main point of
Sourcer is that you can just define the thing that you really want, and then get
on with your life. &lt;/rant&gt;


## Features

- Supports Python version 3.6 and later.
- Create parsers at runtime, or generate Python source code as part of your build.
- Implements [Parsing Expression Grammars](http://en.wikipedia.org/wiki/Parsing_expression_grammar)
  (where "|" represents ordered choice).
- Built-in support for operator precedence parsing.
- Supports inline Python, for defining predicates and transformations directly<commit_msg>Keep fiddling with README.<commit_after># Sourcer

A parsing library for Python.

```python
from sourcer import Grammar

g = Grammar(r'''
    start = "Hello" >> /[a-zA-Z]+/

    ignore Space = /[ \t]+/
    ignore Punctuation = "," | "." | "!" | "?"
''')

assert 'World' == g.parse('Hello, World!')
assert 'Chief' == g.parse('Hello Chief?!?!!')
```

Notes:

* `>>` means "discard the the left hand side"
* `/.../` means "regular expression"


## Installation

Use pip:

```console
$ python3 -m pip install sourcer
```

Notes:

* Sourcer requires Python version 3.6 or later.
* Sourcer is a pure Python library, and has no dependencies.


## Why does this exist?

Sometimes you have to parse things, and sometimes a regex won't cut it.

Things you might have to parse someday:

- log files
- business rules
- market data feeds
- equations
- queries
- user input
- domain specific languages
- obscure data formats
- legacy source code

So that's what this library is for. It's for when you have to take some text
and turn it into a tree of Python objects.


#### But aren't there a ton of parsing libraries for Python already?

Yes, there are. Most of them focus on different problems. Sourcer focuses on the
output of parsing, rather than the means.

The main point of Sourcer is that you can just define the thing that you really
want, and then get on with your life. For example, Sourcer doesn't have a
API for parse trees, because it doesn't need one -- as you'll see in the examples.


## Features

- Supports Python version 3.6 and later.
- Create parsers at runtime, or generate Python source code as part of your build.
- Implements [Parsing Expression Grammars](http://en.wikipedia.org/wiki/Parsing_expression_grammar)
  (where "|" represents ordered choice).
- Built-in support for operator precedence parsing.
- Supports inline Python, for defining predicates and transformations directly
{'fraction_non_alphanumeric': 0.10492081447963801, 'fraction_numerical': 0.002828054298642534, 'mean_word_length': 3.7222963951935912, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3416871', 'n_tokens_mistral': 1079, 'n_tokens_neox': 1008, 'n_words': 474}