<commit_before>==========================
Using the pyparsing module
==========================

:author: <NAME>
:address: <EMAIL>

:revision: 3.0.1
:date: August, 2021

:copyright: Copyright |copy| 2003-2021 Paul McGuire.

.. |copy| unicode:: 0xA9

:abstract: This document provides how-to instructions for the
    pyparsing library, an easy-to-use Python module for constructing
    and executing basic text parsers.  The pyparsing module is useful
    for evaluating user-definable
    expressions, processing custom application language commands, or
    extracting data from formatted reports.

.. sectnum::    :depth: 4

.. contents::   :depth: 4

Note: While this content is still valid, there are more detailed
descriptions and extensive examples at the `online doc server
<https://pyparsing-docs.readthedocs.io/en/latest/pyparsing.html>`_, and
in the online help for the various pyparsing classes and methods (viewable
using the Python interpreter's built-in ``help()`` function). You will also
find many example scripts in the `examples <https://github.com/pyparsing/pyparsing/tree/master/examples>`_
directory of the pyparsing GitHub repo.

Note: In pyparsing 3.0, many method and function names which were
originally written using camelCase have been converted to PEP8-compatible
snake_case. So ``parse_string()`` is being renamed to ``parse_string()``, 
``delimited_list`` to ``delimited_list``, and so on. You may see the old
names in legacy parsers, and they will be supported for a time with
synonyms, but the synonyms will be removed in a future release.

Steps to follow
===============

To parse an incoming data string, the client code must follow these steps:

1. First define the tokens and patterns to be matched, and assign
   this to a program variable.  Optional results names or parse
   actions can also be defined at this time.

2. Call ``parse_string()`` or ``scan_string()`` on this variable, passing in
   the string to
   be parsed.  During the matching process, whitespace between
   tokens is skipped by default (although this can be changed).
   When token matches occur, any defined parse action methods are
   called.

3. Process the parsed results, returned as a ParseResults object.
   The ParseResults object can be accessed as if it were a list of
   strings. Matching results may also be accessed as named attributes of
   the returned results, if names are defined in the definition of
   the token pattern, using ``set_results_name()``.

<commit_msg>Fix old method names in comparison (#295)<commit_after>==========================
Using the pyparsing module
==========================

:author: <NAME>
:address: <EMAIL>

:revision: 3.0.1
:date: August, 2021

:copyright: Copyright |copy| 2003-2021 <NAME>.

.. |copy| unicode:: 0xA9

:abstract: This document provides how-to instructions for the
    pyparsing library, an easy-to-use Python module for constructing
    and executing basic text parsers.  The pyparsing module is useful
    for evaluating user-definable
    expressions, processing custom application language commands, or
    extracting data from formatted reports.

.. sectnum::    :depth: 4

.. contents::   :depth: 4

Note: While this content is still valid, there are more detailed
descriptions and extensive examples at the `online doc server
<https://pyparsing-docs.readthedocs.io/en/latest/pyparsing.html>`_, and
in the online help for the various pyparsing classes and methods (viewable
using the Python interpreter's built-in ``help()`` function). You will also
find many example scripts in the `examples <https://github.com/pyparsing/pyparsing/tree/master/examples>`_
directory of the pyparsing GitHub repo.

Note: In pyparsing 3.0, many method and function names which were
originally written using camelCase have been converted to PEP8-compatible
snake_case. So ``parseString()`` is being renamed to ``parse_string()``, 
``delimitedList`` to ``delimited_list``, and so on. You may see the old
names in legacy parsers, and they will be supported for a time with
synonyms, but the synonyms will be removed in a future release.

Steps to follow
===============

To parse an incoming data string, the client code must follow these steps:

1. First define the tokens and patterns to be matched, and assign
   this to a program variable.  Optional results names or parse
   actions can also be defined at this time.

2. Call ``parse_string()`` or ``scan_string()`` on this variable, passing in
   the string to
   be parsed.  During the matching process, whitespace between
   tokens is skipped by default (although this can be changed).
   When token matches occur, any defined parse action methods are
   called.

3. Process the parsed results, returned as a ParseResults object.
   The ParseResults object can be accessed as if it were a list of
   strings. Matching results may also be accessed as named attributes of
   the returned results, if names are defined in the definition of
   the token pattern, using ``set_results_name()``.


{'fraction_non_alphanumeric': 0.09404952076677316, 'fraction_numerical': 0.010583067092651758, 'mean_word_length': 4.374463519313305, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5907394', 'n_tokens_mistral': 1402, 'n_tokens_neox': 1320, 'n_words': 663}