<commit_before>Enduro/X Machine Independent network cluster protocol
=====================================================
:doctype: book

== About manual

This document describes the network data encoding protocol used for connecting
Enduro/X instances between machines of different architectures. Enduro/X is
operating with the same type of machine architectures and operating systems, then
native format network protocol is recommended to be used, which is sending direct
C structures between the Enduro/X instances.

How ever in cases when Enduro/X application is run on different type of machines
and cluster link is needed between those instances, machine independent format
shall be used. This format can activated by setting flag "-f" in *tpbridge(8)*
command line arguments for both end points.

Protocol is constructed as Tag Length Value (TLV) encoded data, where Tag and Length
has fixed length. Data part may be encoded as Binary Coded Decimal (BCD), ASCII
or Binary data.

Protocol by it self operates in asynchronous fashion and is loosely coupled with
the call semantics (i.e. when the reply is required, etc), it is out of the scope
for protocol it self. How ever this document will present basic work flows for
the messaging sessions.

Currently over the one server:port end-point only one TCP/IP socket is handled.
Each end-point represents a node in Enduro/X cluster with configured / unique 
within cluster Node ID.

== Abbreviations

.Used Abbreviations
|=========================================================
|Abbreviation |Description
|BCD | Binary Coded Decimal. The way how numbers are encoded. The decimal number
is encoded in hex representation. It is more compact than ASCII, but still
manageable to debug in hex dump.
|TLV | Tag Length value
|=========================================================

== Physical network connection

To establish physical network connection with Enduro/X application server, asynchronous
socket shall be used. Each message sent in socket is prefixed with 4 bytes length
indicator. The length bytes does not include length bytes by it self, but it indicates
the number of bytes that are part of the incoming message. After the length indicator
bytes the data bytes follows. Length bytes are order in network order, which is 
big-endian. Thus if message length with out length bytes are in decimal is 15888, 
then following frame bytes shall be sent:

. 0x00

. 0x00

. 0x10

. 0x3e

For keep-a-live, zero length messages are supported. Thus Enduro/X bridge periodically
may be configured to emit the zero length messages. If configured so, bridge will
wait corresponding idle messages too. If not received in configured time, the
connection is restarted.

== Message TLV Encoding rules

Messages are encoded with:

- Two byte TAG which identifies simple data type or complex data structure. Bytes
are encoded in network order big-endian.

- 4 byte (unsigned) data length identifier, also encoded in network order of
big-endian. Length specifies number of is used by data.

- Data block which corresponds to the TAG meaning.

Data alignment is not used for encoded message. When parsing a protocol, any 
un-recognized tags shall be ignored.

== General structures data layout

All the exchanged messages within protocol as is based on the structures principles.
Where we can look on structure like a complex data type. Each structure may
consist of basic data types or complex data types. Which makes messages a
hierarchical tree of the structures. Where leaf structures always consists of
basic data types. This document uses term "block", "complex data typed block", "message blocks" 
and "structures" with the same meaning.

Basically network messages consists of the message blocks. These message blocks
then are constructed basic data types and complex data type blocks.


Typical message layout would be:

image:ex_netproto_msgexample.png[caption="Figure 1: ", title="tpcall() message layout"]

== Basic Data type encoding

This section lists data types used in message protocol and their specifics.


[[numbers-anchor]]
=== Numbers

Numbers in messages are encoded as binary coded decimals. Binary coded decimal
means that value is encoded in ASCII representation of hex value. Thus when analyzing
the message dumps in hex format, the encoded values can be clearly seen. For example
if value of unsigned integer 43219 needs to be transmitted over the protocol,
the number will be encoded in 3 bytes as follows: 0x04,0x32,0x19.

If the number is signed, then last in left-to-right order char encodes the sign. 
"0" is positive or  zero, "1" is negative number. For example having Signed Integer 
type value -717711, it will be encoded as 0x07, 0x17, 0x71, 0x11.

*Floats* and *doubles* are encoded with fixed implied resolution. For Floats 5 digits
decimal fraction is encoded in BCD value. For Doubles 6 digits are encoded in
trailer. So for example, having Double value 654.999812, would be encoded in
BCD as following bytes:x65, x49, x99, 0x81, 0x20 (note as number
is not negative, it is terminated with 0).

Enduro/X may run on 32bit (LP32) and 64bit (LP64) platforms. Some data types differ between
these platforms, particularity *long* and *unsigned long* as they corresponds
to platform bits. Internally Enduro/X uses values which in case
of longs does not go over 32bit values, but user application for example may
send in UBF field 64bit long value. In that particular case C function sscanf()
may overflow if maximum 64bit value is sent to 32bit value. End result is value
which was returned by sscanf().


=== Strings and blobs (Carray)

Strings and blobs are encoded as generic byte array. String data may not contain
0x00 byte, where carray can. The length is encoded in TLV Length component.


[[time-anchor]]
=== Enduro/X time field

Enduro/X internally uses *NTIMER* data type which represents transportation
of number of seconds (20 digits) and number of nanoseconds (20 digits). Thus
Total length of the field is 40 digits, which in BCD is 20 bytes. The "time since"
is relate and exact start time source is specified with the particular field
use.


=== Legend of all data type presentation

For numeric types variable length fields length is specified logical digits. 
For signed numeric types this does not include sign digit. Numeric type length 
does not include zero prefix for BCD in case if odd number of digits are present 
in final BCD representation. 

Field length for strings, blobs (carrays) and sub-structures is specified in
number of bytes.

.List of basic data types
|=========================================================
|Type name |Description
|<Type>..X | Field length form 0 to X.
|<Type>X..Y |  Field length from X to Y.
|<Type>X | Fixed field length X.
|SHORT | Signed short field. 16bit value.
|LONG | Signed long field. 32bit or 64bit value.
|CHAR | One byte ASCII char. Shall not contain 0x00 terminator byte. In that
case field TLV length shall be 0.
|FLOAT | Floating point value. See <<numbers-anchor>> for encoding rules.
|DOUBLE | Double precision value. See <<numbers-anchor>> for encoding rules.
|STRING | String value. May contains all ASCII characters, except 0x00 byte.
|CARRAY | This is blob type, may contain any bytes.
|INT | Signed integer type, 32bit type.
|ULONG | Unsigned long. 32 or 64 bit value.
|UINT | 32bit usinged integer.
|NTIMER | Time field. seconds and nano-seconds. See <<time-anchor>>. Field is
fixed length of 40 digits.
|USHORT | 16bit unsigned value.
|=========================================================


=== Field optionality

All fields in all blocks (unless field is array) are mandatory. However default
value (which usually is 0) is specified for each of the tags, if any. Thus in
that particular case, if field is not received form network, the default value
is used.


== Complex data type blocks

This section list complex data type blocks which are later incorporated in the
message blocks.

=== Network call block
...

=== Standard call block
...

=== Multi-buffer block (array of) for XATMI call data transport

As Enduro/X may send several buffers with one request, for reasons of call info
data, primary buffer and and primary buffer is UBF it might have embedded VIEW,
UBF objects or UBF might contain pointer to other buffers. Thus multiple XATMI
buffers are serialized.

Multi-buffer block is array of 0..N XATMI buffer typed blocks. Each Multi-buffer
block encodes MBUF tag. Which by it self holds the information about what type
of XATMI buffer it represents. And any additional flags, such as is this Multi-buffer
block a call info. Or is it Primary buffer, or a UBF pointer to buffer (i.e.
virtual pointer).

Layout of the block is following:

image:ex_netproto_multibuf.png[caption="Figure 2: ", title="Multi-buffer array"]

.Multi-buffer block
|=========================================================
|TLV TAG |Name |Format |Default value| Description
|0x132F |tag |UINT1..10 |N/A | This is Multi-buffer tag. Not to confuse with
TLV tag. This tag is used to identify the particular Multi-buffer. The tag
consists of first 26 bits of this 32bit unsigned-integer. If bit No *27.* is set
to *1*. This means that particular buffer is *tpsetcallinfo(3)* associated buffer
with primary buffer. The call info bit must be set only for tag *0*. If call
info bit is set, then call primary buffer is at tag *1*. If call info bit 27 is
not set, then primary buffer is at tag *0*. Any other tags are virtual pointer,
i.e. primary buffer in that case must be *UBF* typed and it must hold a *BFLD_PTR*
with references to these tags. Buffer type is by it self is encoded at bits 28..32.
Currently following buffer types are supported: *0* - *UBF* buffer, *2* - *TPINIT*
buffer, *3* - *NULL* buffer, *4* - *STRING* buffer, *5* - *CARRAY* buffer, *6* - 
*JSON* buffer, *7* - *VIEW* buffer.
|=========================================================

=== XATMI buffer
...

==== UBF data
...

===== Embedded VIEW
...

===== Embedded UBF
...

==== View Data
...

==== Other data buffers
...

===== String data
...

===== JSON data
...

===== NULL buffer
...

== Message blocks
...

=== General work flow

Establishing connection - time adjustment

=== Time adjustment exchange block
...

=== Service table refresh block
...

=== Generic command call block
...

=== TP Call / TP Reply block
...

=== TP Notify / TP Broadcast block
...

////////////////////////////////////////////////////////////////
The index is normally left completely empty, it's contents being
generated automatically by the DocBook toolchain.
////////////////////////////////////////////////////////////////
<commit_msg>Update ex_netproto.adoc<commit_after>Enduro/X Machine Independent network cluster protocol
=====================================================
:doctype: book

== About manual

This document describes the network data encoding protocol used for connecting
Enduro/X instances between machines of different architectures. Enduro/X is
operating with the same type of machine architectures and operating systems, then
native format network protocol is recommended to be used, which is sending direct
C structures between the Enduro/X instances.

How ever in cases when Enduro/X application is run on different type of machines
and cluster link is needed between those instances, machine independent format
shall be used. This format can activated by setting flag "-f" in *tpbridge(8)*
command line arguments for both end points.

Protocol is constructed as Tag Length Value (TLV) encoded data, where Tag and Length
has fixed length. Data part may be encoded as Binary Coded Decimal (BCD), ASCII
or Binary data.

Protocol by it self operates in asynchronous fashion and is loosely coupled with
the call semantics (i.e. when the reply is required, etc), it is out of the scope
for protocol it self. How ever this document will present basic work flows for
the messaging sessions.

Currently over the one server:port end-point only one TCP/IP socket is handled.
Each end-point represents a node in Enduro/X cluster with configured / unique 
within cluster Node ID.

== Abbreviations

.Used Abbreviations
|=========================================================
|Abbreviation |Description
|BCD | Binary Coded Decimal. The way how numbers are encoded. The decimal number
is encoded in hex representation. It is more compact than ASCII, but still
manageable to debug in hex dump.
|TLV | Tag Length value
|=========================================================

== Physical network connection

To establish physical network connection with Enduro/X application server, asynchronous
socket shall be used. Each message sent in socket is prefixed with 4 bytes length
indicator. The length bytes does not include length bytes by it self, but it indicates
the number of bytes that are part of the incoming message. After the length indicator
bytes the data bytes follows. Length bytes are order in network order, which is 
big-endian. Thus if message length with out length bytes are in decimal is 15888, 
then following frame bytes shall be sent:

. 0x00

. 0x00

. 0x10

. 0x3e

For keep-a-live, zero length messages are supported. Thus Enduro/X bridge periodically
may be configured to emit the zero length messages. If configured so, bridge will
wait corresponding idle messages too. If not received in configured time, the
connection is restarted.

== Message TLV Encoding rules

Messages are encoded with:

- Two byte TAG which identifies simple data type or complex data structure. Bytes
are encoded in network order big-endian.

- 4 byte (unsigned) data length identifier, also encoded in network order of
big-endian. Length specifies number of is used by data.

- Data block which corresponds to the TAG meaning.

Data alignment is not used for encoded message. When parsing a protocol, any 
un-recognized tags shall be ignored.

== General structures data layout

All the exchanged messages within protocol as is based on the structures principles.
Where we can look on structure like a complex data type. Each structure may
consist of basic data types or complex data types. Which makes messages a
hierarchical tree of the structures. Where leaf structures always consists of
basic data types. This document uses term "block", "complex data typed block", "message blocks" 
and "structures" with the same meaning.

Basically network messages consists of the message blocks. These message blocks
then are constructed basic data types and complex data type blocks.


Typical message layout would be:

image:ex_netproto_msgexample.png[caption="Figure 1: ", title="tpcall() message layout"]

== Basic Data type encoding

This section lists data types used in message protocol and their specifics.


[[numbers-anchor]]
=== Numbers

Numbers in messages are encoded as binary coded decimals. Binary coded decimal
means that value is encoded in ASCII representation of hex value. Thus when analyzing
the message dumps in hex format, the encoded values can be clearly seen. For example
if value of unsigned integer 43219 needs to be transmitted over the protocol,
the number will be encoded in 3 bytes as follows: 0x04,0x32,0x19.

If the number is signed, then last in left-to-right order char encodes the sign. 
"0" is positive or  zero, "1" is negative number. For example having Signed Integer 
type value -717711, it will be encoded as 0x07, 0x17, 0x71, 0x11.

*Floats* and *doubles* are encoded with fixed implied resolution. For Floats 5 digits
decimal fraction is encoded in BCD value. For Doubles 6 digits are encoded in
trailer. So for example, having Double value 654.999812, would be encoded in
BCD as following bytes:x65, x49, x99, 0x81, 0x20 (note as number
is not negative, it is terminated with 0).

Enduro/X may run on 32bit (LP32) and 64bit (LP64) platforms. Some data types differ between
these platforms, particularity *long* and *unsigned long* as they corresponds
to platform bits. Internally Enduro/X uses values which in case
of longs does not go over 32bit values, but user application for example may
send in UBF field 64bit long value. In that particular case C function sscanf()
may overflow if maximum 64bit value is sent to 32bit value. End result is value
which was returned by sscanf().


=== Strings and blobs (Carray)

Strings and blobs are encoded as generic byte array. String data may not contain
0x00 byte, where carray can. The length is encoded in TLV Length component.


[[time-anchor]]
=== Enduro/X time field

Enduro/X internally uses *NTIMER* data type which represents transportation
of number of seconds (20 digits) and number of nanoseconds (20 digits). Thus
Total length of the field is 40 digits, which in BCD is 20 bytes. The "time since"
is relate and exact start time source is specified with the particular field
use.


=== Legend of all data type presentation

For numeric types variable length fields length is specified logical digits. 
For signed numeric types this does not include sign digit. Numeric type length 
does not include zero prefix for BCD in case if odd number of digits are present 
in final BCD representation. 

Field length for strings, blobs (carrays) and sub-structures is specified in
number of bytes.

.List of basic data types
|=========================================================
|Type name |Description
|<Type>..X | Field length form 0 to X.
|<Type>X..Y |  Field length from X to Y.
|<Type>X | Fixed field length X.
|SHORT | Signed short field. 16bit value.
|LONG | Signed long field. 32bit or 64bit value.
|CHAR | One byte ASCII char. Shall not contain 0x00 terminator byte. In that
case field TLV length shall be 0.
|FLOAT | Floating point value. See <<numbers-anchor>> for encoding rules.
|DOUBLE | Double precision value. See <<numbers-anchor>> for encoding rules.
|STRING | String value. May contains all ASCII characters, except 0x00 byte.
|CARRAY | This is blob type, may contain any bytes.
|INT | Signed integer type, 32bit type.
|ULONG | Unsigned long. 32 or 64 bit value.
|UINT | 32bit usinged integer.
|NTIMER | Time field. seconds and nano-seconds. See <<time-anchor>>. Field is
fixed length of 40 digits.
|USHORT | 16bit unsigned value.
|=========================================================


=== Field optionality

All fields in all blocks (unless field is array) are mandatory. However default
value (which usually is 0) is specified for each of the tags, if any. Thus in
that particular case, if field is not received form network, the default value
is used.


== Complex data type blocks

This section list complex data type blocks which are later incorporated in the
message blocks.

=== Network call block
...

=== Standard call block
...

=== Multi-buffer block (array of) for XATMI call data transport

*Complex type block code: MBUF*

As Enduro/X may send several buffers with one request, for reasons of call info
data, primary buffer and and primary buffer is UBF it might have embedded VIEW,
UBF objects or UBF might contain pointer to other buffers. Thus multiple XATMI
buffers are serialized.

Multi-buffer block is array of 0..N XATMI buffer typed blocks. Each Multi-buffer
block encodes MBUF tag. Which by it self holds the information about what type
of XATMI buffer it represents. And any additional flags, such as is this Multi-buffer
block a call info. Or is it Primary buffer, or a UBF pointer to buffer (i.e.
virtual pointer).

Layout of the block is following:

image:ex_netproto_multibuf.png[caption="Figure 2: ", title="Multi-buffer array"]

.Multi-buffer block
|=========================================================
|TLV TAG |Name |Format |Default value| Description
|0x132F |tag |UINT1..10 |N/A | This is Multi-buffer tag. Not to confuse with
TLV tag. This tag is used to identify the particular Multi-buffer. The tag
consists of first 26 bits of this 32bit unsigned-integer. If bit No *27.* is set
to *1*. This means that particular buffer is *tpsetcallinfo(3)* associated buffer
with primary buffer. The call info bit must be set only for tag *0*. If call
info bit is set, then call primary buffer is at tag *1*. If call info bit 27 is
not set, then primary buffer is at tag *0*. Any other tags are virtual pointer,
i.e. primary buffer in that case must be *UBF* typed and it must hold a *BFLD_PTR*
with references to these tags. Buffer type is by it self is encoded at bits 28..32.
Currently following buffer types are supported: *0* - *UBF* buffer, *2* - *TPINIT*
buffer, *3* - *NULL* buffer, *4* - *STRING* buffer, *5* - *CARRAY* buffer, *6* - 
*JSON* buffer, *7* - *VIEW* buffer.
|=========================================================

=== XATMI buffer
...

==== UBF data
...

===== Embedded VIEW
...

===== Embedded UBF
...

==== View Data
...

==== Other data buffers
...

===== String data
...

===== JSON data
...

===== NULL buffer
...

== Message blocks
...

=== General work flow

Establishing connection - time adjustment

=== Time adjustment exchange block
...

=== Service table refresh block
...

=== Generic command call block
...

=== TP Call / TP Reply block
...

=== TP Notify / TP Broadcast block
...

////////////////////////////////////////////////////////////////
The index is normally left completely empty, it's contents being
generated automatically by the DocBook toolchain.
////////////////////////////////////////////////////////////////

{'fraction_non_alphanumeric': 0.1089206274877078, 'fraction_numerical': 0.015546710372278155, 'mean_word_length': 4.045121663123081, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4501618', 'n_tokens_mistral': 5763, 'n_tokens_neox': 5307, 'n_words': 3100}