<commit_before># Workflow Service

## Overview
The pz-workflow service enables the construction and use of "event" notifications to enable simple "if-this-happens-then-do-that" workflows. This is done though an HTTP API. For a walkthough visit the Users Guide, for a simple overview visit `https://pz-swagger.geointservices.io/#/Workflow`.

A user will follow these general steps:

. Register a new event type
. Register a trigger for that event type
. Send an event
. Poll for new alerts
. Go to 3.

## Running Locally
To find out how to run the pz-workflow service locally, please visit the github [README](https://github.com/venicegeo/pz-workflow/blob/readme-updates/README.md)

## Technical

### Elasticsearch Interaction
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html[Elasticsearch] is the chosen database for the workflow service. Elasticsearch can be defined as a 'nosql' using Lucene query. This can provide powerful query ability but at the cost of some inconveniences interacting with the database. When running locally workflow will automatically attach to a local elasticsearch instance, but when using an offsite elasticsearch addition information must be provided in the VCAP services.
Workflow has been designed in a way to reduce elasticsearch's 'nosql' capabilites. This is done by creating strict mapping types on all schemas. This insures that elasticsearch will only accept documents that exactly match the specified schema.
Workflow uses two packages in order to communicate with elasticsearch. The first of these is https://github.com/venicegeo/pz-gocommon/blob/master/elasticsearch/index.go[pz-gocommon index file]. This allows workflow to create simple communication with an index. This index file itself relies on http://gopkg.in/olivere/elastic.v3[elastic.v3] to create the HTTP traffic.

### Kafka Interaction
Information on Kafka can be found https://kafka.apache.org/intro[here].
In workflow, Kafka service information must be provided in the VCAP services.
After an event 'triggers' a trigger, a job is sent to kafka. The function for this can be found https://github.com/venicegeo/pz-workflow/blob/ff8b869893f910145d5205ed2557f22ca0e1da24/workflow/Service.go#L206[here].
Often when running workflow locally VCAP services are not provided, therefore no connection with kafka can be made, making jobs fail to be sent. A way of testing whether or not triggers work is checking to see if a kafka error was created in the workflow process terminal.

### Other Required Services

### Programmatic Hierarchy

// vim: set syntax=asciidoc wrap:<commit_msg>Update 15-pz-workflow.txt<commit_after># Workflow Service

## Overview
The pz-workflow service enables the construction and use of "event" notifications to enable simple "if-this-happens-then-do-that" workflows. This is done though an HTTP API. For a walkthough visit the Users Guide, for a simple overview visit `https://pz-swagger.geointservices.io/#/Workflow`.

A user will follow these general steps:

. Register a new event type
. Register a trigger for that event type
. Send an event
. Poll for new alerts
. Go to 3.

## Running Locally
To find out how to run the pz-workflow service locally, please visit the github [README](https://github.com/venicegeo/pz-workflow/blob/readme-updates/README.md)

## Technical

### Elasticsearch Interaction
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html[Elasticsearch] is the chosen database for the workflow service. Elasticsearch can be defined as a 'nosql' using Lucene query. This can provide powerful query ability but at the cost of some inconveniences interacting with the database. When running locally workflow will automatically attach to a local elasticsearch instance, but when using an offsite elasticsearch addition information must be provided in the VCAP services.
Workflow has been designed in a way to reduce elasticsearch's 'nosql' capabilites. This is done by creating strict mapping types on all schemas. This insures that elasticsearch will only accept documents that exactly match the specified schema.
Workflow uses two packages in order to communicate with elasticsearch. The first of these is https://github.com/venicegeo/pz-gocommon/blob/master/elasticsearch/index.go[pz-gocommon index file]. This allows workflow to create simple communication with an index. This index file itself relies on http://gopkg.in/olivere/elastic.v3[elastic.v3] to create the HTTP traffic.

### Kafka Interaction
Information on Kafka can be found https://kafka.apache.org/intro[here].
In workflow, Kafka service information must be provided in the VCAP services.
After an event 'triggers' a trigger, a job is sent to kafka. The function for this can be found https://github.com/venicegeo/pz-workflow/blob/ff8b869893f910145d5205ed2557f22ca0e1da24/workflow/Service.go#L206[here].
Often when running workflow locally VCAP services are not provided, therefore no connection with kafka can be made, making jobs fail to be sent. A way of testing whether or not triggers work is checking to see if a kafka error was created in the workflow process terminal.

### Other Required Services
pz-workflow also requires two other services: pz-idam & pz-servicecontroller. These do not require the use of VCAP as they are part of the piazza system. Workflow will only fail to launch if it cannot contact the piazza system. Service controller is used in workflow to validate service uuids when creating a trigger. Found https://github.com/venicegeo/pz-workflow/blob/3864890c012854223569212af85d36eb077d1725/workflow/TriggerDB.go#L54[here]. Idam is used when a trigger is attempting to fire off a job, which idam can allow or deny. Found https://github.com/venicegeo/pz-workflow/blob/ac6a9287aea9e9ec24a83a38e1b19895a14df730/workflow/Service.go#L782[here], however notice that the actual auth function is contained in https://github.com/venicegeo/pz-gocommon/blob/master/gocommon/authz.go[pz-gocommon].

### Programmatic Hierarchy
The workflow service has three distinct hierarchical sections: `Server`, `Service`, `DB`. The Server is the top layer. This is where HTTP requests are forwarded two. The job of the `Server` functions is to call the correct function in the `Service` and return a valid json response. The `Service` contains the bulk of the operation. There are functions designed for support, such as `sendToKafka` or `newIdent`, and functions that mirror those in the `Server`. These functions perform the logic of whichever rest call was made, interacting with `DB`'s to do so. The `DB`'s interact with the index mentioned in the elasticsearch interaction section. The role of these functions is to provide higher level functions when communicating with elasticsearch, such as queries.

// vim: set syntax=asciidoc wrap:
{'fraction_non_alphanumeric': 0.06441176470588235, 'fraction_numerical': 0.01911764705882353, 'mean_word_length': 4.8629310344827585, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 15, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5547718', 'n_tokens_mistral': 1897, 'n_tokens_neox': 1784, 'n_words': 860}