<commit_before># Protocol Documentation

## Binary Protocol is not final!  There may be breaking changes at anytime.  Contact me if you wish to write a driver.

### Binary Protocol

The binary protocol is similar to the json protocol, just a different encoding, meant to be faster to encode and decode then json for high performance critical applications (databases, queues, ect). Please read the json protocol spec for more details, as this doc simply describes the encoding. One of the main goals of the binary protocol is make the sharding proxy faster (see goshire-shards project)



Request Format

* All integers are signed big endian.
* Variable length fields (arrays and strings) are all prefixed by a 'length' int16
* All length fields are int16, unless otherwise noted
* All strings are utf8 encoded

### Hello Packet

Clients should send this packet on initial connection.  No response will be sent from the server.  Server will disconnect without a proper hello

```
[encoding (int8)] //same values as param_encoding, currently should always be 0 (json) 
[length][json hello (string)]
```

The hello json package should look like the following:

```
{
    "v" : 2.0 //the protocol version (required)
    "useragent" : //the useragent (required)
    "service" : //the service (if used to connect to a shard router only)
}
```

After hello is sent, then the client is free to start sending requests, and recieving responses.
The protocol is async (same as the json protocol), so clients should always listen for new responses.  There are only 2 packet types: REQUEST and RESPONSE.  


### REQUEST

```
[partition (int16)] //set to -1 as default
[length][shard key (string)] 
[router table revision (int64)]

[length][txn_id (string)]
[txn_accept(int8)]
[method(int8)]
[length][uri (string)]
[param_encoding (int8)]
[length][params (array)]
[content_encoding (int8)]
[content_length (int32)][content (array)]
```
 -- Note the shard key and partition sections are only used for goshire-shard router requests. Other requests should zero those fields


### RESPONSE

```
[length][txn_id (string)]
[txn_status(int8)]
[status (int16)]
[length][status_message (string)]
[content_encoding (int8)]
[content_length (int32)][content (array)]
```
<commit_msg>update for params map in response, this would be additional to the normal content array<commit_after># Protocol Documentation

## Binary Protocol is not final!  There may be breaking changes at anytime.  Contact me if you wish to write a driver.

### Binary Protocol

The binary protocol is similar to the json protocol, just a different encoding, meant to be faster to encode and decode then json for high performance critical applications (databases, queues, ect). Please read the json protocol spec for more details, as this doc simply describes the encoding. One of the main goals of the binary protocol is make the sharding proxy faster (see goshire-shards project)



Request Format

* All integers are signed big endian.
* Variable length fields (arrays and strings) are all prefixed by a 'length' int16
* All length fields are int16, unless otherwise noted
* All strings are utf8 encoded

### Hello Packet

Clients should send this packet on initial connection.  No response will be sent from the server.  Server will disconnect without a proper hello

```
[encoding (int8)] //same values as param_encoding, currently should always be 0 (json) 
[length][json hello (string)]
```

The hello json package should look like the following:

```
{
    "v" : 2.0 //the protocol version (required)
    "useragent" : //the useragent (required)
    "service" : //the service (if used to connect to a shard router only)
}
```

After hello is sent, then the client is free to start sending requests, and recieving responses.
The protocol is async (same as the json protocol), so clients should always listen for new responses.  There are only 2 packet types: REQUEST and RESPONSE.  


### REQUEST

```
[partition (int16)] //set to -1 as default
[length][shard key (string)] 
[router table revision (int64)]

[length][txn_id (string)]
[txn_accept(int8)]
[method(int8)]
[length][uri (string)]
[param_encoding (int8)]
[length][params (array)]
[content_encoding (int8)]
[content_length (int32)][content (array)]
```
 -- Note the shard key and partition sections are only used for goshire-shard router requests. Other requests should zero those fields


### RESPONSE

```
[length][txn_id (string)]
[txn_status(int8)]
[status (int16)]
[length][status_message (string)]
[param_encoding (int8)]
[length][params (array)]
[content_encoding (int8)]
[content_length (int32)][content (array)]
```

{'fraction_non_alphanumeric': 0.09424196678887212, 'fraction_numerical': 0.011861117101574293, 'mean_word_length': 4.761490683229813, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2596595', 'n_tokens_mistral': 1339, 'n_tokens_neox': 1266, 'n_words': 620}