<commit_before>.. _taskmodule_simplestats:

SimpleStats
-----------

TODO<commit_msg>Add documentation for SimpleStats task module<commit_after>.. _taskmodule_simplestats:

SimpleStats
-----------

The ``SimpleStats`` task module is a :ref:`periodic_tasks` to collect some basic statistics
from the token database and write them to the time series database table ``MonitoringStats``.

Options
~~~~~~~

The ``SimpleStats`` task module provides the following boolean options:

**total_tokens**

    If activated, the total number of tokens in the token database will be
    monitored.


**hardware_tokens**

    If activated, the total number of hardware tokens in the token database will
    be monitored.


**software_tokens**

    If activated, the total number of software tokens in the token database will
    be monitored.


**unassigned_hardware_tokens**

    If activated, the number of hardware tokens in the token database which are
    not assigned to a user will be monitored.


**assigned_tokens**

    If activated, the number of tokens in the token database which are assigned
    to users will be monitored.


**user_with_token**

    If activated, the number of users which have at least one token assigned
    will be monitored.

.. note:: The statistics key, with which the time series is identified in the
    ``MonitoringStats`` table, is the same as the option name.

    Adding a statistic with the same key to the ``MonitoringStats`` table will
    corrupt the data.

.. note:: For each of these basic statistic values the token database will be
    queried. To avoid excessive database load, consider running the
    ``SimpleStats`` task infrequently.
{'fraction_non_alphanumeric': 0.07287933094384708, 'fraction_numerical': 0.0, 'mean_word_length': 3.731638418079096, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '341623', 'n_tokens_mistral': 440, 'n_tokens_neox': 408, 'n_words': 214}