<commit_before># nes Protocol v0.5.x

## Message

The nes protocol consists of JSON messages sent between the client and server.

Each incoming request from the client to the server contains:
- `type` - the message type:
    - `'ping'` - heartbeat response.
    - `'hello'` - connection initialization and authentication.
    - `'request'` - endpoint request.
    - `'sub'` - subscribe to a path.
    - `'unsub'` - unsubscribe from a path.
    - `'message'` - send custom message.
- additional type-specific fields.

Each outgoing request from the server to the client contains:
- `type` - the message type:
    - `'ping'` - heartbeat request.
    - `'hello'` - connection initialization and authentication.
    - `'request'` - endpoint request.
    - `'sub'` - subscribe to a path.
    - `'message'` - send custom message.
    - `'broadcast'` - a broadcast message to all clients.
    - `'pub'` - a subscription update.
- additional type-specific fields.

## Errors

When a message indicates an error, the message will include in addition to the message-specific fields:
- `statusCode` - an HTTP equivalent status code (4xx, 5xx).
- `headers` - optional headers related to the request.
- `payload` - the error details which include:
    - `error` - the HTTP equivalent error message.
    - `message` - a description of the error.
    - additional error-specific fields.

For example:

```js
{
    type: 'hello',
    id: 1,
    statusCode: 401,
    payload: {
        error: 'Unauthorized',
        message: 'Unknown username or incorrect password'
    }
}
```

## Heartbeat

Flow: `server` -> `client` -> `server`

For cases where it is not possible for the TCP connection to determine if the connection is still active,
the server sends a heartbeat message to the client every configured interval, and then expects the client
to respond within a configured timeout. The server sends:
- `type` - set to `'ping'`.

For example:

```js
{
    type: 'ping'
}
```

When the client receives the message, it sends back:
- `type` - set to `'ping'`.
- `id` - a unique per-client request id (number or string).

For example:

```js
{
    type: 'ping',
    id: 6
}
```

## Hello

Flow: `client` -> `server` -> `client`

Every client connection must first be initialized with a `hello` message. The client sends a message to the server
with the following:
- `type` - set to `'hello'`.
- `id` - a unique per-client request id (number or string).
- `auth` - optional authentication credentials. Can be any value understood by the server.
- `subs` - an optional array of strings indicating the path subscriptions the client is interested in.

For example:

```js
{
    type: 'hello',
    id: 1,
    auth: {
        headers: {
            authorization: 'Basic <KEY>
        }
    },
    subs: ['/a', '/b']
}
```

The server respond by sending a message back with the following:
- `type` - set to `'hello'`.
- `id` - the same `id` received from the client.
- `heartbeat` - the server heartbeat configuration which can be:
    - `false` - no heartbeats will be sent.
    - an object with:
        - `interval` - the heartbeat interval in milliseconds.
        - `timeout` - the time from sending a heartbeat to the client until a response is expected
          before a connection is considered closed by the server.

Note: the client should assume the connection is closed if it has not heard from the server in
heartbeat.interval + heartbeat.timeout.

For example:

```js
{
    type: 'hello',
    id: 1
}
```

If the request failed, the server includes the [standard error fields](#errors).

For example:

```js
{
    type: 'hello',
    id: 1,
    statusCode: 401,
    payload: {
        error: 'Unauthorized',
        message: 'Unknown username or incorrect password'
    }
}
```

If any of the subscription requests fail, the server will send separate messages before the `'hello'` response:
- `type` - set to `'sub'`.
- `path` - the requested path which failed to subscribe.
- the [standard error fields](#errors)

For example:<commit_msg>heartbeat example<commit_after># nes Protocol v0.5.x

## Message

The nes protocol consists of JSON messages sent between the client and server.

Each incoming request from the client to the server contains:
- `type` - the message type:
    - `'ping'` - heartbeat response.
    - `'hello'` - connection initialization and authentication.
    - `'request'` - endpoint request.
    - `'sub'` - subscribe to a path.
    - `'unsub'` - unsubscribe from a path.
    - `'message'` - send custom message.
- `id` - a unique per-client request id (number or string).
- additional type-specific fields.

Each outgoing request from the server to the client contains:
- `type` - the message type:
    - `'ping'` - heartbeat request.
    - `'hello'` - connection initialization and authentication.
    - `'request'` - endpoint request.
    - `'sub'` - subscribe to a path.
    - `'message'` - send custom message.
    - `'broadcast'` - a broadcast message to all clients.
    - `'pub'` - a subscription update.
- additional type-specific fields.

## Errors

When a message indicates an error, the message will include in addition to the message-specific fields:
- `statusCode` - an HTTP equivalent status code (4xx, 5xx).
- `headers` - optional headers related to the request.
- `payload` - the error details which include:
    - `error` - the HTTP equivalent error message.
    - `message` - a description of the error.
    - additional error-specific fields.

For example:

```js
{
    type: 'hello',
    id: 1,
    statusCode: 401,
    payload: {
        error: 'Unauthorized',
        message: 'Unknown username or incorrect password'
    }
}
```

## Heartbeat

Flow: `server` -> `client` -> `server`

For cases where it is not possible for the TCP connection to determine if the connection is still active,
the server sends a heartbeat message to the client every configured interval, and then expects the client
to respond within a configured timeout. The server sends:
- `type` - set to `'ping'`.

For example:

```js
{
    type: 'ping'
}
```

When the client receives the message, it sends back:
- `type` - set to `'ping'`.
- `id` - a unique per-client request id (number or string).

For example:

```js
{
    type: 'ping',
    id: 6
}
```

## Hello

Flow: `client` -> `server` -> `client`

Every client connection must first be initialized with a `hello` message. The client sends a message to the server
with the following:
- `type` - set to `'hello'`.
- `id` - a unique per-client request id (number or string).
- `auth` - optional authentication credentials. Can be any value understood by the server.
- `subs` - an optional array of strings indicating the path subscriptions the client is interested in.

For example:

```js
{
    type: 'hello',
    id: 1,
    auth: {
        headers: {
            authorization: 'Basic am9objpzZWNyZXQ='
        }
    },
    subs: ['/a', '/b']
}
```

The server respond by sending a message back with the following:
- `type` - set to `'hello'`.
- `id` - the same `id` received from the client.
- `heartbeat` - the server heartbeat configuration which can be:
    - `false` - no heartbeats will be sent.
    - an object with:
        - `interval` - the heartbeat interval in milliseconds.
        - `timeout` - the time from sending a heartbeat to the client until a response is expected
          before a connection is considered closed by the server.

Note: the client should assume the connection is closed if it has not heard from the server in
heartbeat.interval + heartbeat.timeout.

For example:

```js
{
    type: 'hello',
    id: 1,
    heartbeat: {
        interval: 15000,
        timeout: 5000
    }
}
```

If the request failed, the server includes the [standard error fields](#errors).

For example:

```js
{
    type: 'hello',
    id: 1,
    statusCode: 401,
    payload: {
        error: 'Unauthorized',
        message: 'Unknown username or incorrect password'
    }
}
```

If any of the subscription requests fail, the server will send separate messages before the `'hello'` response:
- `type` - set to `'sub'`.
- `path` - the requested path which failed to subscribe.
- the [standard error fields](#errors)

For example:
{'fraction_non_alphanumeric': 0.11203218334755577, 'fraction_numerical': 0.004876264781177618, 'mean_word_length': 2.9366602687140113, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5542630', 'n_tokens_mistral': 2418, 'n_tokens_neox': 2334, 'n_words': 1034}