<commit_before>## Prerequisites for using piazza

In order to use piazza in full or its individual apps assuming other parts are running, the following are needed:

    Java Development Kit 1.8

    Maven 3.3.9

    Oracle VM VirtualBox  5.0.10 (for running the services such as kafka/elastic/mongo/postgis/geoserver on local individual VMs)

    Vagrant (to manage services VMs)

	MongoDB 2.6.3
	
	GeoServer 2.9.2
	
	PostgreSQL (Crunchy) 9.5.2
	
	ElasticSearch 2.2
	
	Kafka 0.9.0.1


### Gateway API

The Gateway API is the single point-of-contact for external users to interact with Piazza Core components. This API provides functionality for loading and accessing data, registering and executing services, and registering events and triggers. The entirety of Piazza functionality is accessed through these Gateway API calls.

See the Swagger documentation for a full explanation of all of these endpoints.


#### Authentication

Authentication with the Gateway is done using basic authentication. This means setting the authentication header in your request to basic authentication, which takes in the username and password and base64 encodes it into a string, which is passed along to the gateway and decoded. These credentials are then passed on to `pz-idam` security component which handles authentication and authorization.


#### Jobs

For most requests handled through the Piazza Gateway, the response will contain the expected information for said request (for example - querying data will return a list of data results, etc.), however: when executing a request within Piazza that could take a long amount of time (such as ingesting data, or executing an external service) Piazza will return a Job ID which can be used to track the status and progress of that request. The motivation for providing a Job ID is to allow the Gateway to return a response immediately, which will not block the user. It will also allow the user to track the progress of a long-running job as it is handled by internal Piazza components.

Endpoints that return a Job ID are documented and outlined in the Swagger documentation.

[source,json]
----
{
    "jobId": "my-job-id"
}
----

The response may look something like the above. In this case, the requesting user can then take the Job ID and re-query the Gateway with the Job ID in order to get the latest status and progress of that Job.

When the Job has been completed, the result of the associated Job (being a data ID, or service, or whatever the end result of the Job was) will be contained in the status of that Job, including the time it was completed and how long it took.
<commit_msg>Adding table for local vagrant services<commit_after>## Prerequisites for using piazza

In order to use piazza in full or its individual apps assuming other parts are running, the following are needed:

    Java Development Kit 1.8

    Maven 3.3.9

    Oracle VM VirtualBox  5.0.10 (for running the services such as kafka/elastic/mongo/postgis/geoserver on local individual VMs)

    Vagrant (to manage services VMs)

	MongoDB 2.6.3
	
	GeoServer 2.9.2
	
	PostgreSQL (Crunchy) 9.5.2
	
	ElasticSearch 2.2
	
	Kafka 0.9.0.1
	
	Local vagrant boxes available for services:
	| Vagrant Box   |      Location      |
	|---------------|:--------------------------------------------------------------------------:|
	| ElasticSearch |  https://github.com/venicegeo/pz-search-metadata-ingest/tree/master/config |
	| MongoDB       |  https://github.com/venicegeo/pz-jobmanager/tree/master/config |
	| GeoServer     |  https://github.com/venicegeo/pz-access/tree/master/config |
	| PostgreSQL    |  https://github.com/venicegeo/pz-ingest/tree/master/config |
	| Kafka         |  https://github.com/venicegeo/kafka-devbox |


### Gateway API

The Gateway API is the single point-of-contact for external users to interact with Piazza Core components. This API provides functionality for loading and accessing data, registering and executing services, and registering events and triggers. The entirety of Piazza functionality is accessed through these Gateway API calls.

See the Swagger documentation for a full explanation of all of these endpoints.


#### Authentication

Authentication with the Gateway is done using basic authentication. This means setting the authentication header in your request to basic authentication, which takes in the username and password and base64 encodes it into a string, which is passed along to the gateway and decoded. These credentials are then passed on to `pz-idam` security component which handles authentication and authorization.


#### Jobs

For most requests handled through the Piazza Gateway, the response will contain the expected information for said request (for example - querying data will return a list of data results, etc.), however: when executing a request within Piazza that could take a long amount of time (such as ingesting data, or executing an external service) Piazza will return a Job ID which can be used to track the status and progress of that request. The motivation for providing a Job ID is to allow the Gateway to return a response immediately, which will not block the user. It will also allow the user to track the progress of a long-running job as it is handled by internal Piazza components.

Endpoints that return a Job ID are documented and outlined in the Swagger documentation.

[source,json]
----
{
    "jobId": "my-job-id"
}
----

The response may look something like the above. In this case, the requesting user can then take the Job ID and re-query the Gateway with the Job ID in order to get the latest status and progress of that Job.

When the Job has been completed, the result of the associated Job (being a data ID, or service, or whatever the end result of the Job was) will be contained in the status of that Job, including the time it was completed and how long it took.

{'fraction_non_alphanumeric': 0.06393470498214589, 'fraction_numerical': 0.008842033667743582, 'mean_word_length': 3.6645519429024582, 'pattern_counts': {'":': 2, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5424713', 'n_tokens_mistral': 1545, 'n_tokens_neox': 1424, 'n_words': 836}