<commit_before>== Metrics

The following is an extensive list of metrics (counters, timers, gauges, ...)
published organically by Genie, in addition to metrics published by Spring, JVM
and system metrics and statistics.

Metrics are collected using http://micrometer.io/[Micrometer] which allows system admins to plugin a variety of backend
collection systems (Atlas, Datadog, Graphite, Ganglia, etc). See http://micrometer.io/[website] for more details. Genie
ships with no backend system compiled in. It will have to be added if one is desired otherwise metrics are just
published within the local JVM and available on the Actuator `/metrics` endpoint.

[cols=">1m,<10,^1,^1m,^1m",options="header"]
|===
|Name |Description |Unit |Source |Tags

|genie.jobs.submit.timer
|Time taken initialize the job environment (working directory, script) and fork the children
|nanoseconds
|JobLauncher
|status, exceptionClass

|genie.jobs.tasks.applicationTask.timer
|Time taken to stage all applications that a job depends on
|nanoseconds
|ApplicationTask
|status, applicationId, exceptionClass

|genie.jobs.tasks.applicationTask.applicationSetup.timer
|Time taken to set up individual applications (creating folders, staging dependencies and configurations)
|nanoseconds
|ApplicationTask
|applicationId, applicationName

|genie.jobs.tasks.clusterTask.timer
|Time taken to set up cluster a job runs on (creating folders, staging dependencies and configurations)
|nanoseconds
|ClusterTask
|clusterId, clusterName, status, exceptionClass<commit_msg>Document new agent connected metric<commit_after>== Metrics

The following is an extensive list of metrics (counters, timers, gauges, ...)
published organically by Genie, in addition to metrics published by Spring, JVM
and system metrics and statistics.

Metrics are collected using http://micrometer.io/[Micrometer] which allows system admins to plugin a variety of backend
collection systems (Atlas, Datadog, Graphite, Ganglia, etc). See http://micrometer.io/[website] for more details. Genie
ships with no backend system compiled in. It will have to be added if one is desired otherwise metrics are just
published within the local JVM and available on the Actuator `/metrics` endpoint.

[cols=">1m,<10,^1,^1m,^1m",options="header"]
|===
|Name |Description |Unit |Source |Tags

|genie.agents.connected.gauge
|The number of agents connected to the server
|count
|AgentMetricsServiceImpl
|-

|genie.jobs.submit.timer
|Time taken initialize the job environment (working directory, script) and fork the children
|nanoseconds
|JobLauncher
|status, exceptionClass

|genie.jobs.tasks.applicationTask.timer
|Time taken to stage all applications that a job depends on
|nanoseconds
|ApplicationTask
|status, applicationId, exceptionClass

|genie.jobs.tasks.applicationTask.applicationSetup.timer
|Time taken to set up individual applications (creating folders, staging dependencies and configurations)
|nanoseconds
|ApplicationTask
|applicationId, applicationName

|genie.jobs.tasks.clusterTask.timer
|Time taken to set up cluster a job runs on (creating folders, staging dependencies and configurations)
|nanoseconds
|ClusterTask
|clusterId, clusterName, status, exceptionClass
{'fraction_non_alphanumeric': 0.07891453524641298, 'fraction_numerical': 0.0037429819089207735, 'mean_word_length': 6.126666666666667, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5819773', 'n_tokens_mistral': 886, 'n_tokens_neox': 842, 'n_words': 360}