<commit_before>*******
repchar
*******

This is a simple command line script that creates a list of all
characters contained in a given document.

Usage
=====

repchar [-h] -o OUTFILENAME [-v] [-c] INFILENAME

positional arguments:
  INFILENAME            Filename

optional arguments:
  -h, --help            show this help message and exit
  -o OUTFILENAME, --outfile OUTFILENAME, --out OUTFILENAME
                        Filename for report
  -v, --verbose         Set log level to INFO.
  --version             show program's version number and exit
  -c, --comb, --combining
                        Create special report for combining characters

Module
======

You can also use the ``CharReporter`` class in your own code:

.. code:: python

    >>> from repchar import CharReporter
    >>> s = u"ABCD"
    >>> r = CharReporter()
    >>> r.feed(s)
    >>> r.report()
    chr	hex	count	unicode
    A	0041	1	LATIN CAPITAL LETTER A
    B	0042	1	LATIN CAPITAL LETTER B
    C	0043	1	LATIN CAPITAL LETTER C
    D	0044	1	LATIN CAPITAL LETTER D

The count of each letter is stored in ``CharReporter.chars``:

.. code:: python

    >>> r.chars
    {u'A': 1, u'C': 1, u'B': 1, u'D': 1}
<commit_msg>Test with pycon for syntax highlighting.<commit_after>*******
repchar
*******

This is a simple command line script that creates a list of all
characters contained in a given document.

Usage
=====

repchar [-h] -o OUTFILENAME [-v] [-c] INFILENAME

positional arguments:
  INFILENAME            Filename

optional arguments:
  -h, --help            show this help message and exit
  -o OUTFILENAME, --outfile OUTFILENAME, --out OUTFILENAME
                        Filename for report
  -v, --verbose         Set log level to INFO.
  --version             show program's version number and exit
  -c, --comb, --combining
                        Create special report for combining characters

Module
======

You can also use the ``CharReporter`` class in your own code:

.. code:: pycon

    >>> from repchar import CharReporter
    >>> s = u"ABCD"
    >>> r = CharReporter()
    >>> r.feed(s)
    >>> r.report()
    chr	hex	count	unicode
    A	0041	1	LATIN CAPITAL LETTER A
    B	0042	1	LATIN CAPITAL LETTER B
    C	0043	1	LATIN CAPITAL LETTER C
    D	0044	1	LATIN CAPITAL LETTER D

The count of each letter is stored in ``CharReporter.chars``:

.. code:: pycon

    >>> r.chars
    {u'A': 1, u'C': 1, u'B': 1, u'D': 1}

{'fraction_non_alphanumeric': 0.1113871635610766, 'fraction_numerical': 0.01987577639751553, 'mean_word_length': 2.3416320885200554, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3451748', 'n_tokens_mistral': 852, 'n_tokens_neox': 752, 'n_words': 307}