<commit_before>#+TITLE:     EP Stats
#+AUTHOR:    <NAME>
#+EMAIL:     <EMAIL>
#+DATE:      2010-02-08 Mon
#+DESCRIPTION:
#+KEYWORDS:
#+LANGUAGE:  en
#+OPTIONS:   H:3 num:t toc:t \n:nil @:t ::t |:t ^:nil -:t f:t *:t <:t
#+OPTIONS:   TeX:t LaTeX:nil skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+INFOJS_OPT: view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js
#+EXPORT_SELECT_TAGS: export
#+EXPORT_EXCLUDE_TAGS: noexport
#+LINK_UP:
#+LINK_HOME:
#+STYLE:  <link rel="stylesheet" type="text/css" href="myorg.css" />


* Stats Definitions

| Stat                          | Description                              |
|-------------------------------+------------------------------------------|
| ep_storage_age                | Seconds since most recently              |
|                               | stored object was initially queued.      |
| ep_storage_age_highwat        | ep_storage_age high water mark           |
| ep_min_data_age               | Minimum data age setting.                |
| ep_queue_age_cap              | Queue age cap setting.                   |
| ep_data_age                   | Second since most recently               |
|                               | stored object was modified.              |
| ep_data_age_highwat           | ep_data_age high water mark              |
| ep_too_young                  | Number of times an object was            |
|                               | not stored due to being too young.       |
| ep_too_old                    | Number of times an object was            |
|                               | stored after being dirty too long.       |
| ep_queue_size                 | Number of items queued for storage.      |
| ep_flusher_todo               | Number of items remaining to be written. |
| ep_commit_time                | Number of seconds of most recent commit. |
| ep_flush_duration             | Number of seconds of most recent flush.  |
| ep_flush_duration_highwat     | ep_flush_duration high water mark.       |
| ep_dbname                     | DB path.                                 |
| ep_dbinit                     | Number of seconds to initialize DB.      |
| ep_warmup                     | true if warmup is enabled.               |
| ep_warmup_thread              | Warmup thread status.                    |
| ep_warmed_up                  | Number of items warmed up.               |
| ep_warmup_time                | Number of seconds spent warming data.    |
| eq_tapq:client_id:qlen        | Queue size for the given client_id.      |
| eq_tapq:client_id:rec_fetched | Tap messages sent to the client.         |
| ep_tap_total_queue            | Sum of tap queue sizes.                  |
| ep_tap_total_fetched          | Sum of tap messages sent.                |
| ep_tap_keepalive              | Tap keepalive time.                      |

* Details

** Ages

The difference between =ep_storage_age= and =ep_data_age= is somewhat
subtle, but when you consider that a given record may be updated
multiple times before hitting persistence, it starts to be clearer.

=ep_data_age= is how old the data we actually wrote is.

=ep_storage_age= is how long the object has been waiting to be
persisted.
<commit_msg>Bug 980 - Improved stats description<commit_after>#+TITLE:     EP Stats
#+AUTHOR:    <NAME>
#+EMAIL:     <EMAIL>
#+DATE:      2010-02-08 Mon
#+DESCRIPTION:
#+KEYWORDS:
#+LANGUAGE:  en
#+OPTIONS:   H:3 num:t toc:t \n:nil @:t ::t |:t ^:nil -:t f:t *:t <:t
#+OPTIONS:   TeX:t LaTeX:nil skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+INFOJS_OPT: view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js
#+EXPORT_SELECT_TAGS: export
#+EXPORT_EXCLUDE_TAGS: noexport
#+LINK_UP:
#+LINK_HOME:
#+STYLE:  <link rel="stylesheet" type="text/css" href="myorg.css" />


* Stats Definitions

| Stat                          | Description                              |
|-------------------------------+------------------------------------------|
| ep_storage_age                | Seconds since most recently              |
|                               | stored object was initially queued.      |
| ep_storage_age_highwat        | ep_storage_age high water mark           |
| ep_min_data_age               | Minimum data age setting.                |
| ep_queue_age_cap              | Queue age cap setting.                   |
| ep_data_age                   | Second since most recently               |
|                               | stored object was modified.              |
| ep_data_age_highwat           | ep_data_age high water mark              |
| ep_too_young                  | Number of times an object was            |
|                               | not stored due to being too young.       |
| ep_too_old                    | Number of times an object was            |
|                               | stored after being dirty too long.       |
| ep_queue_size                 | Number of items queued for storage.      |
| ep_flusher_todo               | Number of items remaining to be written. |
| ep_commit_time                | Number of seconds of most recent commit. |
| ep_flush_duration             | Number of seconds of most recent flush.  |
| ep_flush_duration_highwat     | ep_flush_duration high water mark.       |
| ep_dbname                     | DB path.                                 |
| ep_dbinit                     | Number of seconds to initialize DB.      |
| ep_warmup                     | true if warmup is enabled.               |
| ep_warmup_thread              | Warmup thread status.                    |
| ep_warmed_up                  | Number of items warmed up.               |
| ep_warmup_time                | Number of seconds spent warming data.    |
| eq_tapq:client_id:qlen        | Queue size for the given client_id.      |
| eq_tapq:client_id:rec_fetched | Tap messages sent to the client.         |
| ep_tap_total_queue            | Sum of tap queue sizes on the current    |
|                               | tap queues                               |
| ep_tap_total_fetched          | Sum of tap messages sent on the current  |
|                               | tap queues                               |
| ep_tap_keepalive              | Tap keepalive time.                      |

* Details

** Ages

The difference between =ep_storage_age= and =ep_data_age= is somewhat
subtle, but when you consider that a given record may be updated
multiple times before hitting persistence, it starts to be clearer.

=ep_data_age= is how old the data we actually wrote is.

=ep_storage_age= is how long the object has been waiting to be
persisted.

{'fraction_non_alphanumeric': 0.09543071161048688, 'fraction_numerical': 0.003445692883895131, 'mean_word_length': 1.4535097390665197, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4688603', 'n_tokens_mistral': 1921, 'n_tokens_neox': 1807, 'n_words': 566}