<commit_before>Syslog Processor
================

A framework for processing syslog messages using Python


Handlers
--------

Handlers are created by subclassing handler.LogEntryHandler and placing a
module in the handlers directory. Each subclass of LogEntryHandler in this
directory is instanciated once.

Handlers pass filters for the entries they are interested to their parent class
in their constructor. See the example handlers and handler.py for more
information.


Filters:
* privals - A bitmask of priotity flags. See syslog documentation. This is bitmasked against the message prival to determine if the handler should be called.
* timestamp - ISO formatted timestamp
* hostname - Name of host which created the message
* app\_name - Name of app which created the message
* procid - ID (numeral) of process which created the message
* msgid - A numeric ID for the message
* msg - Message data

Additionally, filters can be marked as not disjunctive (making them conjunctive). By default filters are disjunctive. When in disjunctive form, the filters can be thought of as a list of boolean values (whether the filter matches the message) or'd together. When in conjunctive form the filters are combined using and's.


Configuration
-------------

Run syslogprocessor -h for argument information

<commit_msg>Updated README with notes about default filter values<commit_after>Syslog Processor
================

A framework for processing syslog messages using Python


Handlers
--------

Handlers are created by subclassing handler.LogEntryHandler and placing a
module in the handlers directory. Each subclass of LogEntryHandler in this
directory is instanciated once.

Handlers pass filters for the entries they are interested to their parent class
in their constructor. See the example handlers and handler.py for more
information.


Filters:
* privals - A bitmask of priotity flags. See syslog documentation. This is bitmasked against the message prival to determine if the handler should be called.
* timestamp - ISO formatted timestamp
* hostname - Name of host which created the message
* app\_name - Name of app which created the message
* procid - ID (numeral) of process which created the message
* msgid - A numeric ID for the message
* msg - Message data

Additionally, filters can be marked as not disjunctive (making them conjunctive). By default filters are disjunctive. When in disjunctive form, the filters can be thought of as a list of boolean values (whether the filter matches the message) or'd together. When in conjunctive form the filters are combined using and's. When in disjunctive form a filter set to None represents a filter which matches no values. When in conjunctive form a filter set to none represents a filter that matches all values.


Configuration
-------------

Run syslogprocessor -h for argument information


{'fraction_non_alphanumeric': 0.056643356643356645, 'fraction_numerical': 0.0, 'mean_word_length': 4.220802919708029, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2782790', 'n_tokens_mistral': 686, 'n_tokens_neox': 646, 'n_words': 401}