<commit_before>.. _re_worker_output:

RE-WORKER-OUTPUT
----------------
Release Engine Worker Plugin consumes output from other works on the bus and writes it to files.

.. note::
   This is an output consumer and is not meant to be used in steps.


This worker takes two configuration files. The first is the worker configuration file. It should look like this:

.. code-block:: json

  {
      "queue": "output",
      "output_dir": "/tmp/",
      "redactions': [
          "somethingtomakealineasredacted",
          "valid[rR]?egex"
      ]
  }

* Set the ``output_dir`` to where the output files should reside.

.. versionadded:: 0.0.2
* ``redactions`` is an optional config option. If set it should be a list of strings or valid regular expressions which will be used to find lines that will be redacted.<commit_msg>redactions config in reworkeroutput had wrong quoting.<commit_after>.. _re_worker_output:

RE-WORKER-OUTPUT
----------------
Release Engine Worker Plugin consumes output from other works on the bus and writes it to files.

.. note::
   This is an output consumer and is not meant to be used in steps.


This worker takes two configuration files. The first is the worker configuration file. It should look like this:

.. code-block:: json

  {
      "queue": "output",
      "output_dir": "/tmp/",
      "redactions": [
          "somethingtomakealineasredacted",
          "valid[rR]?egex"
      ]
  }

* Set the ``output_dir`` to where the output files should reside.

.. versionadded:: 0.0.2
* ``redactions`` is an optional config option. If set it should be a list of strings or valid regular expressions which will be used to find lines that will be redacted.
{'fraction_non_alphanumeric': 0.1026706231454006, 'fraction_numerical': 0.0035608308605341245, 'mean_word_length': 2.939252336448598, 'pattern_counts': {'":': 5, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '894549', 'n_tokens_mistral': 476, 'n_tokens_neox': 458, 'n_words': 213}