<commit_before># Workflow Service

Piazza provides the ability for users to define and send _Events_, representing some "thing" that has happened. These Events can be issued either from within Piazza or by an external client. Piazza also allows for users to define _Triggers_, containing both an event-condition and an action: when the condition is met, the action is performed. Taken together, _EventTypes_, _Events_, and _Triggers_ can be used to define _workflows_.

NOTE: By design, Events and Triggers are fairly primitive, low-level constructs. As Piazza matures, richer support for defining workflows may be added.

## The EventType

The user (or some service) first defines an _EventType_ which is the "schema" for the Events that the user will be generating. The _EventType_ object is sent in a `POST` request to the `/eventType` endpoint and contains contains a unique `name` (string) and a mapping describing the EventType's parameters. For example:

[source,json]
----
{
    "name": "testevent-1468849135",
    "mapping": {
        "filename": "string",
        "code":     "string",
        "severity": "integer"
    }
}
----

The `name` must be unique across all event types in the system.

The available data types are `string`, `boolean`, `integer`, `double`, `date`, `float`, `byte`, `short`, and `long`. By no coincidence, these are the basic types that Elasticsearch supports.

The following script shows an example of registering an EventType. Run it using:

    $ ./7-eventtype.sh

link:scripts/7-eventtype.sh[7-eventtype.sh]
[source,bash]
----
include::scripts/7-eventtype.sh[tags=public]
----


## The Trigger

The user then defines a _Trigger_ which defines what action is to be taken when a specific event occurs. The _Trigger_ is sent as a `POST` request to the `/trigger` endpoint and contains four parts. The `condition` defines what type of event is to be watched for and what the specific parameters of that event should be, expressed using Elasticsearch DSL query syntax against the parameters in the event type. The `job` defines what action is to be taken, expressed as using Gateway/Dispatcher-style syntax. The `title` is a memorable string for describing what the _Trigger_ is meant to do, and the `enabled` field determines if the _Trigger_ should be listening for Events in order to send Alerts. For example:

[source,json]
----
{
    "title": "High Severity",
    "enabled": true,
    "condition": {
        "eventTypeIds": ["98fc25e8-bd97-4444-a972-c06aa0f0edf1"],
        "query": {
            "query": {
                "bool": {
                    "must": [
                        { "match": {"severity": 5} },
                        { "match": {"code": "PHONE"} }
                    ]
                }
            }
        }
    },
    "job": {
        "createdBy": "test",
        "jobType": {
            "type": "execute-service",
            "data": {
                "serviceId": "a2898bcb-2646-4ffd-9da7-2308cb7e77d7"
            }
        }
    }
}
----

For details on constructing valid Elasticsearch DSL queries, see the <<Elasticsearch Query Syntax>> section.

In this example, the job will be executed only when our "test" event occurs with the `severity` equal to `5` and the `code` equal to `"PHONE"`.

Pass the following script an eventTypeId, and it will create a generic Trigger for the EventType associated with that eventTypeId:

    $ ./7-trigger.sh {{eventTypeId}}

link:scripts/7-trigger.sh[7-trigger.sh]
[source,bash]
----
include::scripts/7-trigger.sh[tags=public]
----
<commit_msg>Add explanation of using event data in trigger<commit_after># Workflow Service

Piazza provides the ability for users to define and send _Events_, representing some "thing" that has happened. These Events can be issued either from within Piazza or by an external client. Piazza also allows for users to define _Triggers_, containing both an event-condition and an action: when the condition is met, the action is performed. Taken together, _EventTypes_, _Events_, and _Triggers_ can be used to define _workflows_.

NOTE: By design, Events and Triggers are fairly primitive, low-level constructs. As Piazza matures, richer support for defining workflows may be added.

## The EventType

The user (or some service) first defines an _EventType_ which is the "schema" for the Events that the user will be generating. The _EventType_ object is sent in a `POST` request to the `/eventType` endpoint and contains contains a unique `name` (string) and a mapping describing the EventType's parameters. For example:

[source,json]
----
{
    "name": "testevent-1468849135",
    "mapping": {
        "filename": "string",
        "code":     "string",
        "severity": "integer"
    }
}
----

The `name` must be unique across all event types in the system.

The available data types are `string`, `boolean`, `integer`, `double`, `date`, `float`, `byte`, `short`, and `long`. By no coincidence, these are the basic types that Elasticsearch supports.

The following script shows an example of registering an EventType. Run it using:

    $ ./7-eventtype.sh

link:scripts/7-eventtype.sh[7-eventtype.sh]
[source,bash]
----
include::scripts/7-eventtype.sh[tags=public]
----


## The Trigger

The user then defines a _Trigger_ which defines what action is to be taken when a specific event occurs. The _Trigger_ is sent as a `POST` request to the `/trigger` endpoint and contains four parts. The `condition` defines what type of event is to be watched for and what the specific parameters of that event should be, expressed using Elasticsearch DSL query syntax against the parameters in the event type. The `job` defines what action is to be taken, expressed as using Gateway/Dispatcher-style syntax. The `title` is a memorable string for describing what the _Trigger_ is meant to do, and the `enabled` field determines if the _Trigger_ should be listening for Events in order to send Alerts. For example:

[source,json]
----
{
    "title": "High Severity",
    "enabled": true,
    "condition": {
        "eventTypeIds": ["98fc25e8-bd97-4444-a972-c06aa0f0edf1"],
        "query": {
            "query": {
                "bool": {
                    "must": [
                        { "match": {"severity": 5} },
                        { "match": {"code": "PHONE"} }
                    ]
                }
            }
        }
    },
    "job": {
        "jobType": {
            "type": "execute-service",
            "data": {
                "serviceId": "a2898bcb-2646-4ffd-9da7-2308cb7e77d7",
                "dataInputs": {
                    "test": {
                        "content": "{ \"log\": \"Received code $code with severity $severity\" }",
                        "type": "body",
                        "mimeType": "application/json"
                    }
                },
                "dataOutput": [ {
                    "mimeType":"image/tiff",
                    "type":"raster"
                } ]
            }
        }
    }
}
----

For details on constructing valid Elasticsearch DSL queries, see the <<Elasticsearch Query Syntax>> section.

In this example, the job will be executed only when our "test" event occurs with the `severity` equal to `5` and the `code` equal to `"PHONE"`.

It's important to note that the "job" field uses substitution by replacing all instances of `$field` (where `field` is the name of a JSON field in the EventType `mapping` (and thus subsequently in the Event's `data` field)) with the `field` in the event that sets off the Trigger. This substitution occurs in all of the fields in `job`, so it is important to be conscious of this.

Pass the following script an `eventTypeId`, and it will create a generic Trigger for the EventType associated with that `eventTypeId`:

    $ ./7-trigger.sh {{eventTypeId}}

link:scripts/7-trigger.sh[7-trigger.sh]
[source,bash]
----
include::scripts/7-trigger.sh[tags=public]
----

{'fraction_non_alphanumeric': 0.10054176641048256, 'fraction_numerical': 0.01461509386418042, 'mean_word_length': 2.7926421404682276, 'pattern_counts': {'":': 54, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2997001', 'n_tokens_mistral': 2251, 'n_tokens_neox': 2139, 'n_words': 937}