<commit_before>
Runnable API
============

eHive exposes an interface for Runnables (jobs) to interact with the
system:

  - query their own parameters. See :ref:`parameters-in-jobs`
  - control its own execution and report issues
  - run system commands
  - trigger some *dataflow* events (e.g. create new jobs)


Reporting and logging
---------------------

Jobs can log messages to the standard output with the
``$self->say_with_header($message, $important)`` method. However they are only printed
when the *debug* mode is enabled (see below) or when the ``$important`` flag is switched on.
They will also be prefixed with a standard prefix consisting of the
runtime context (worker, role, job).

The debug mode is controlled by the ``--debug X`` option of
:ref:`script-beekeeper` and :ref:`script-runWorker`. *X* is an integer,
allowing multiple levels of debug, although most of the modules will only
check whether it is 0 or not.

``$self->warning($message)`` calls ``$self->say_with_header($message, 1)``
(so that the messages are printed on the standard output) but also stores
them in the database (in the ``log_message`` table).

To indicate that a job has to be terminated earlier (i.e. before reaching
the end of ``write_output``), you can call:

- ``$self->complete_early($message)`` to mark the job as *DONE*
  (successful run). Beware that this will trigger the *autoflow*.
- ``$self->complete_early($message, $branch_code)`` is a variation of the
  above that will replace the autoflow (branch 1) with a dataflow on the
  branch given
- ``$self->throw($message)`` to log a failed attempt. The job may be given
  additional retries following the analysis' *max_retry_count* parameter,
  or is marked as *FAILED* in the database.

System interactions
-------------------

All Runnables have access to the ``$self->run_system_command`` method to run
arbitrary system commands (the ``SystemCmd`` Runnable is merely a wrapper
around this method).

``run_system_command`` takes two arguments:

#. The command to run, given as a single string or an arrayref. Arrayrefs
   are the preferred way as they simplify the handling of whitespace and
   quotes in the command-line arguments. Arrayrefs that correspond to
   straightforward commands, e.g. ``['find', '-type', 'd']``, are passed to
   the underlying ``system`` function as lists. Arrayrefs can contain shell<commit_msg>Should tell that complete_early writes the message in the database<commit_after>
Runnable API
============

eHive exposes an interface for Runnables (jobs) to interact with the
system:

  - query their own parameters. See :ref:`parameters-in-jobs`
  - control its own execution and report issues
  - run system commands
  - trigger some *dataflow* events (e.g. create new jobs)


Reporting and logging
---------------------

Jobs can log messages to the standard output with the
``$self->say_with_header($message, $important)`` method. However they are only printed
when the *debug* mode is enabled (see below) or when the ``$important`` flag is switched on.
They will also be prefixed with a standard prefix consisting of the
runtime context (worker, role, job).

The debug mode is controlled by the ``--debug X`` option of
:ref:`script-beekeeper` and :ref:`script-runWorker`. *X* is an integer,
allowing multiple levels of debug, although most of the modules will only
check whether it is 0 or not.

``$self->warning($message)`` calls ``$self->say_with_header($message, 1)``
(so that the messages are printed on the standard output) but also stores
them in the database (in the ``log_message`` table).

To indicate that a job has to be terminated earlier (i.e. before reaching
the end of ``write_output``), you can call:

- ``$self->complete_early($message)`` to mark the job as *DONE*
  (successful run) and record the message in the database. Beware that this
  will trigger the *autoflow*.
- ``$self->complete_early($message, $branch_code)`` is a variation of the
  above that will replace the autoflow (branch 1) with a dataflow on the
  branch given
- ``$self->throw($message)`` to log a failed attempt. The job may be given
  additional retries following the analysis' *max_retry_count* parameter,
  or is marked as *FAILED* in the database.

System interactions
-------------------

All Runnables have access to the ``$self->run_system_command`` method to run
arbitrary system commands (the ``SystemCmd`` Runnable is merely a wrapper
around this method).

``run_system_command`` takes two arguments:

#. The command to run, given as a single string or an arrayref. Arrayrefs
   are the preferred way as they simplify the handling of whitespace and
   quotes in the command-line arguments. Arrayrefs that correspond to
   straightforward commands, e.g. ``['find', '-type', 'd']``, are passed to
   the underlying ``system`` function as lists. Arrayrefs can contain shell
{'fraction_non_alphanumeric': 0.10983134512546278, 'fraction_numerical': 0.0012340600575894694, 'mean_word_length': 4.201069518716578, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4810396', 'n_tokens_mistral': 1331, 'n_tokens_neox': 1303, 'n_words': 657}