<commit_before>---
layout: default
title: Getting Started
last_updated: Last modified by <NAME> on July 1, 2016
---

# Getting Started

## Deploying the Job Service with Chateau

**[Chateau](https://github.hpe.com/caf/chateau)** can launch workers and services, such as the Job Service.

- To download and set up Chateau, follow the instructions in the [README.md](https://github.hpe.com/caf/chateau/blob/develop/README.md). 

- Installation instructions for the Job Service database can be found [here](https://github.hpe.com/caf/chateau/blob/develop/services/job-service/README.md).

- To deploy the Job Service and job tracking worker, follow the [Service Deployment](https://github.hpe.com/caf/chateau/blob/develop/deployment.md) guide and use the following option with the deployment shell script.

  `./deploy-service.sh job-service`

## Using the Job Service Web User Interface

A handy user interface is provided on the same host and port as the web service. The Swagger user interface page is accessible from the following address:

`<job-service-web-service-url>/job-service-ui`

### Adding a Job

1. Expand the PUT /jobs/{jobId} method. 
2. Enter a value for jobId. 
3. Click on the example value box on the right to fill in the new job body. 
4. Edit these fields with your own details:
 
 `name`: name of the job <br>
  `description`: description of the job <br>
  `externalData`: external data <br>
  `taskClassifier`: classifier of the task <br>
  `taskApiVersion`: API version of the task <br>
  `taskData`: data of the task (include a batch definition if sending to the batch worker) <br>
  `taskDataEncoding`: encoding of the task data, for example, `utf8` <br>
  `taskPipe`: name of the RabbitMQ queue feeding messages to the first worker <br>
  `targetPipe`: name of the final worker's output queue where tracking will stop

5. Press `Try it out!`. The resulting code will show whether the addition of the job succeeds or not. 
   - 201, if the job is successfully added
   - 204, if the job is successfully updated

![Add Job](images/JobServiceUIAddJob.PNG)

### Getting Jobs

1. Expand the GET /jobs method. 
2. Press `Try it out!`. The list of jobs in the system appears in the response body, including the job you just created.

![Add Job](images/JobServiceUIGet.PNG)

## Deploying an End-To-End System

In order to test an end-to-end Job Service system, you need to deploy and run:

- the Job Service (see _Deploying the Job Service with Chateau_)
- a job tracking worker (see _Deploying the Job Service with Chateau_)
- a batch worker 
- another service to send the tasks to, in this case, the example worker.

### Batch Worker

You can deploy the batch worker with Chateau.

Prerequisites for running the batch worker can be found [here](https://github.hpe.com/caf/chateau/blob/develop/services/batch-worker/README.md).

The following command with the deployment script deploys a batch worker:

`./deploy-service.sh batch-worker`

### Example Worker

You can deploy the example worker using Chateau.

Prerequisites for running the example worker can be found [here](https://github.hpe.com/caf/chateau/blob/develop/services/example-worker/README.md).

The following command with the deployment script deploys an example worker:

`./deploy-service.sh example-worker`

You can view the status of the services on Marathon at the following URL:

`<marathon-endpoint>/ui/#`

The figure shows you the health of the workers and services:

![Marathon Health](images/MarathonAllHealthy.png)

You also need dummy data in a datastore and a storage reference to this data. Dummy data can be uploaded from the document-generator. For more information on using the document generator, see the README.md.

### Send a Job

Open the Swagger user interface as explained under _Using the Job Service Web User Interface_.

Add a job with the new job body following this template:

```
{
  "name": "Job_1",
  "description": "end-to-end",
  "externalData": "string",
  "task": {
    "taskClassifier": "BatchWorker",
    "taskApiVersion": 1,
    "taskData": "{\"batchDefinition\":\"[\\\"2f0e1a924d954ed09966f91d726e4960/fda3cf959a1d456b8d54800ba9e9b2f5\\\",\\\"02f0e1a924d954ed09966f91d726e4960/fda3cf959a1d456b8d54800ba9e9b2f5\\\"]\",\"batchType\":\"AssetIdBatchPlugin\",\"taskMessageType\":\"ExampleWorkerTaskBuilder\",\"taskMessageParams\":{\"datastorePartialReference\":\"2f0e1a924d954ed09966f91d726e4960\",\"action\":\"REVERSE\"},\"targetPipe\":\"dataprocessing-example-in\"}",
    "taskDataEncoding": "utf8",
    "taskPipe": "dataprocessing-batch-in",
    "targetPipe": "dataprocessing-example-out"
    
  }
}
```<commit_msg>CAF-1344: Updated Getting-Started.md to link through to job-service readme.md<commit_after>---
layout: default
title: Getting Started
last_updated: Last modified by <NAME> on July 1, 2016
---

# Getting Started

## Deploying the Job Service with Chateau

**[Chateau](https://github.hpe.com/caf/chateau)** can launch workers and services, such as the Job Service.

- To download and set up Chateau, follow the instructions in the [README.md](https://github.hpe.com/caf/chateau/blob/develop/README.md). 

- Installation instructions for the Job Service database can be found [here](https://github.hpe.com/caf/chateau/blob/develop/services/job-service/README.md).

- To deploy the Job Service and job tracking worker, follow the [Service Deployment](https://github.hpe.com/caf/chateau/blob/develop/deployment.md) guide and use the following option with the deployment shell script.

  `./deploy-service.sh job-service`

## Using the Job Service Web User Interface

A handy user interface is provided on the same host and port as the web service. The Swagger user interface page is accessible from the following address:

`<job-service-web-service-url>/job-service-ui`

### Adding a Job

1. Expand the PUT /jobs/{jobId} method. 
2. Enter a value for jobId. 
3. Click on the example value box on the right to fill in the new job body. 
4. Edit these fields with your own details:
 
 `name`: name of the job <br>
  `description`: description of the job <br>
  `externalData`: external data <br>
  `taskClassifier`: classifier of the task <br>
  `taskApiVersion`: API version of the task <br>
  `taskData`: data of the task (include a batch definition if sending to the batch worker) <br>
  `taskDataEncoding`: encoding of the task data, for example, `utf8` <br>
  `taskPipe`: name of the RabbitMQ queue feeding messages to the first worker <br>
  `targetPipe`: name of the final worker's output queue where tracking will stop

5. Press `Try it out!`. The resulting code will show whether the addition of the job succeeds or not. 
   - 201, if the job is successfully added
   - 204, if the job is successfully updated

![Add Job](images/JobServiceUIAddJob.PNG)

### Getting Jobs

1. Expand the GET /jobs method. 
2. Press `Try it out!`. The list of jobs in the system appears in the response body, including the job you just created.

![Add Job](images/JobServiceUIGet.PNG)

## Deploying an End-To-End System

In order to test an end-to-end Job Service system, you need to deploy and run:

- the Job Service (see _Deploying the Job Service with Chateau_)
- a job tracking worker (see _Deploying the Job Service with Chateau_)
- a batch worker 
- another service to send the tasks to, in this case, the example worker.

### Batch Worker

You can deploy the batch worker with Chateau.

Prerequisites for running the batch worker can be found [here](https://github.hpe.com/caf/chateau/blob/develop/services/batch-worker/README.md).

The following command with the deployment script deploys a batch worker:

`./deploy-service.sh batch-worker`

### Example Worker

You can deploy the example worker using Chateau.

Prerequisites for running the example worker can be found [here](https://github.hpe.com/caf/chateau/blob/develop/services/example-worker/README.md).

The following command with the deployment script deploys an example worker:

`./deploy-service.sh example-worker`

You can view the status of the services on Marathon at the following URL:

`<marathon-endpoint>/ui/#`

The figure shows you the health of the workers and services:

![Marathon Health](images/MarathonAllHealthy.png)

You also need dummy data in a datastore and a storage reference to this data. Dummy data can be uploaded from the document-generator. For more information on using the document generator, see the [README.md](https://github.hpe.com/caf/policy-worker/blob/develop/PolicyWorkerDocumentGenerator/README.md).

### Send a Job

Open the Swagger user interface as explained under _Using the Job Service Web User Interface_.

Add a job with the new job body following this template:

```
{
  "name": "Job_1",
  "description": "end-to-end",
  "externalData": "string",
  "task": {
    "taskClassifier": "BatchWorker",
    "taskApiVersion": 1,
    "taskData": "{\"batchDefinition\":\"[\\\"2f0e1a924d954ed09966f91d726e4960/fda3cf959a1d456b8d54800ba9e9b2f5\\\",\\\"02f0e1a924d954ed09966f91d726e4960/fda3cf959a1d456b8d54800ba9e9b2f5\\\"]\",\"batchType\":\"AssetIdBatchPlugin\",\"taskMessageType\":\"ExampleWorkerTaskBuilder\",\"taskMessageParams\":{\"datastorePartialReference\":\"2f0e1a924d954ed09966f91d726e4960\",\"action\":\"REVERSE\"},\"targetPipe\":\"dataprocessing-example-in\"}",
    "taskDataEncoding": "utf8",
    "taskPipe": "dataprocessing-batch-in",
    "targetPipe": "dataprocessing-example-out"
    
  }
}
```
{'fraction_non_alphanumeric': 0.10552550641637501, 'fraction_numerical': 0.02757450418920352, 'mean_word_length': 4.327683615819209, 'pattern_counts': {'":': 34, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 13, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4873650', 'n_tokens_mistral': 3079, 'n_tokens_neox': 2882, 'n_words': 1084}