<commit_before># evp (Event Pipeline)

Listen from JSON Event sources and to process each event
adapting and either forwarding it to an http/https sink or to a mssql database.

Event are processed according to rules defined in a JSON configuration file.

### JSON Dsl

Configuration files are JSON files of the form:

``` json
{
  "sourceName1": {
    "type": "RabbitMQ",
    "config": {
      "host": "192.168.50.4"
    },
    "process": {
      "transform": {
        "a":  "$.nested.a",
        "b":  "$.nested.b"
      },
      "forward": {
        "uri": "http://example.com/dbo/lots"
      }
    }
  },
  "sourceName2": {
    // ...
  }
}
```

### Built in event sources

#### RabbitMQ

``` json
{
  "host": "srv-bln-03",
  "vhost": "hansolo",
  "username": "guest",
  "password": "<PASSWORD>",
  "queue": "data-map-Auctionata.Consignment.Message.Valuation.V1.ValuationRequest",
  "exchange": "Auctionata.Consignment.Message.Valuation.V1.ValuationRequest",
  "routingKey": "frontdoor-valuation-request",
  "permanent": true
}
```

### Built in commands

#### Transform

Applies a transform template to input event and send the result to the next command.

Transform pulls data from original event using JSONPath and generate a new object based on a template.

Each of the template's properties can pull a single property from the source data or an array of all results found by its JSONPath.<commit_msg>cleaned README.md<commit_after># evp (Event Pipeline)

Listen from JSON Event sources and to process each event
adapting and either forwarding it to an http/https sink or to a mssql database.

Event are processed according to rules defined in a JSON configuration file.

### JSON Dsl

Configuration files are JSON files of the form:

``` json
{
  "sourceName1": {
    "type": "RabbitMQ",
    "config": {
      "host": "192.168.50.4"
    },
    "process": {
      "transform": {
        "a":  "$.nested.a",
        "b":  "$.nested.b"
      },
      "forward": {
        "uri": "http://example.com/dbo/lots"
      }
    }
  },
  "sourceName2": {
    // ...
  }
}
```

### Built in commands

#### Transform

Applies a transform template to input event and send the result to the next command.

Transform pulls data from original event using JSONPath and generate a new object based on a template.

Each of the template's properties can pull a single property from the source data or an array of all results found by its JSONPath.
{'fraction_non_alphanumeric': 0.124435318275154, 'fraction_numerical': 0.010677618069815195, 'mean_word_length': 3.046511627906977, 'pattern_counts': {'":': 30, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5023507', 'n_tokens_mistral': 775, 'n_tokens_neox': 743, 'n_words': 276}