<commit_before>Federation API
==============

Matrix home servers use the Federation APIs (also known as server-server APIs)
to communicate with each other.
Home servers use these APIs to push messages to each other in real-time, to
request historic messages from each other, and to query profile and presence
information about users on each other's servers.

The APIs are implemented using HTTPS GETs and PUTs between each of the
servers. These HTTPS requests are strongly authenticated using public key
signatures at the TLS transport layer and using public key signatures in
HTTP Authorization headers at the HTTP layer.

There are three main kinds of communication that occur between home servers:

Persisted Data Units (PDUs):
    These events are broadcast from one home server to any others that have
    joined the same "context" (namely, a Room ID). They are persisted in
    long-term storage and record the history of messages and state for a
    context.

    Like email, it is the responsibility of the originating server of a PDU
    to deliver that event to its recepient servers. However PDUs are signed
    using the originating server's public key so that it is possible to
    deliver them through third-party servers.

Ephemeral Data Units (EDUs):
    These events are pushed between pairs of home servers. They are not
    persisted and are not part of the history of a "context", nor does the
    receiving home server have to reply to them.

Queries:
    These are single request/response interactions between a given pair of
    servers, initiated by one side sending an HTTPS GET request to obtain some
    information, and responded by the other. They are not persisted and contain
    no long-term significant history. They simply request a snapshot state at
    the instant the query is made.


EDUs and PDUs are further wrapped in an envelope called a Transaction, which is
transferred from the origin to the destination home server using an HTTPS PUT
request.

Server Discovery
----------------

Resolving Server Names
~~~~~~~~~~~~~~~~~~~~~~

Each matrix home server is identified by a server name consisting of a DNS name
and an optional TLS port.

.. code::

    server_name = dns_name [ ":" tls_port]
    dns_name = <host, see [RFC 3986], Section 3.2.2>
    tls_port = *DIGIT

.. **

If the port is present then the server is discovered by looking up an A record
for the DNS name and connecting to the specified TLS port. If the port is
absent then the server is discovered by looking up a ``_matrix._tcp``
SRV record for the DNS name.

Home servers may use SRV records to load balance requests between multiple TLS
endpoints or to failover to another endpoint if an endpoint fails.

Retrieving Server Keys
~~~~~~~~~~~~~~~~~~~~~~

Version 2
+++++++++

Each home server publishes its public keys under ``/_matrix/key/v2/server/``.
Home servers query for keys by either getting ``/_matrix/key/v2/server/``
directly or by querying an intermediate perspective server using a
``/_matrix/key/v2/query`` API. Intermediate perspective servers query the
``/_matrix/key/v2/server/`` API on behalf of another server and sign the
response with their own key. A server may query multiple perspective servers
to ensure that they all report the same public keys.

Publishing Keys
_______________

Home servers publish the allowed TLS fingerprints and signing keys in a JSON
object at ``/_matrix/key/v2/server/${key_id}``. The response contains a list of
``verify_keys`` that are valid for signing federation requests made by the
server and for signing events. It contains a list of ``old_verify_keys``
which are only valid for signing events. Finally the response contains a list
of TLS certificate fingerprints to validate any connection made to the server.

A server may have multiple keys active at a given time. A server may have any
number of old keys. It is recommended that servers return a single JSON
response listing all of its keys whenever any ``key_id`` is requested to reduce
the number of round trips needed to discover the relevant keys for a server.
However a server may return a different responses for a different ``key_id``.

The ``tls_certificates`` contain a list of hashes of the X.509 TLS certificates
currently used by the server. The list must include SHA-256 hashes for every
certificate currently in use by the server. These fingerprints are valid until
the millisecond POSIX timestamp in ``valid_until_ts``.

The ``verify_keys`` can be used to sign requests and events made by the server
until the millisecond POSIX timestamp in ``valid_until_ts``. If a Home Server
receives an event with a ``origin_server_ts`` after the ``valid_until_ts`` then
it should request that ``key_id`` for the originating server to check whether
the key has expired.

The ``old_verify_keys`` can be used to sign events with an ``origin_server_ts``
before the ``expired_ts``.

Intermediate perspective servers should cache a response for half of its
remaining life time to avoid serving a stale response. Servers should avoid
querying for certificates more frequently than once an hour to avoid flooding
a server with requests.

==================== =================== ======================================
    Key                    Type                         Description
==================== =================== ======================================
``server_name``      String              DNS name of the home server.
``verify_keys``      Object              Public keys of the home server for
                                         verifying digital signatures.
``old_verify_keys``  Object              The public keys that the server used
                                         to use and when it stopped using them.
``signatures``       Object              Digital signatures for this object
                                         signed using the ``verify_keys``.
``tls_fingerprints`` Array of Objects    Hashes of X.509 TLS certificates used
                                         by this this server encoded as base64.
``valid_until_ts``   Integer             POSIX timestamp when the list of valid
                                         keys should be refreshed.
==================== =================== ======================================


.. code:: json

    {
        "old_verify_keys": {<commit_msg>Describe what the expired_ts is<commit_after>Federation API
==============

Matrix home servers use the Federation APIs (also known as server-server APIs)
to communicate with each other.
Home servers use these APIs to push messages to each other in real-time, to
request historic messages from each other, and to query profile and presence
information about users on each other's servers.

The APIs are implemented using HTTPS GETs and PUTs between each of the
servers. These HTTPS requests are strongly authenticated using public key
signatures at the TLS transport layer and using public key signatures in
HTTP Authorization headers at the HTTP layer.

There are three main kinds of communication that occur between home servers:

Persisted Data Units (PDUs):
    These events are broadcast from one home server to any others that have
    joined the same "context" (namely, a Room ID). They are persisted in
    long-term storage and record the history of messages and state for a
    context.

    Like email, it is the responsibility of the originating server of a PDU
    to deliver that event to its recepient servers. However PDUs are signed
    using the originating server's public key so that it is possible to
    deliver them through third-party servers.

Ephemeral Data Units (EDUs):
    These events are pushed between pairs of home servers. They are not
    persisted and are not part of the history of a "context", nor does the
    receiving home server have to reply to them.

Queries:
    These are single request/response interactions between a given pair of
    servers, initiated by one side sending an HTTPS GET request to obtain some
    information, and responded by the other. They are not persisted and contain
    no long-term significant history. They simply request a snapshot state at
    the instant the query is made.


EDUs and PDUs are further wrapped in an envelope called a Transaction, which is
transferred from the origin to the destination home server using an HTTPS PUT
request.

Server Discovery
----------------

Resolving Server Names
~~~~~~~~~~~~~~~~~~~~~~

Each matrix home server is identified by a server name consisting of a DNS name
and an optional TLS port.

.. code::

    server_name = dns_name [ ":" tls_port]
    dns_name = <host, see [RFC 3986], Section 3.2.2>
    tls_port = *DIGIT

.. **

If the port is present then the server is discovered by looking up an A record
for the DNS name and connecting to the specified TLS port. If the port is
absent then the server is discovered by looking up a ``_matrix._tcp``
SRV record for the DNS name.

Home servers may use SRV records to load balance requests between multiple TLS
endpoints or to failover to another endpoint if an endpoint fails.

Retrieving Server Keys
~~~~~~~~~~~~~~~~~~~~~~

Version 2
+++++++++

Each home server publishes its public keys under ``/_matrix/key/v2/server/``.
Home servers query for keys by either getting ``/_matrix/key/v2/server/``
directly or by querying an intermediate perspective server using a
``/_matrix/key/v2/query`` API. Intermediate perspective servers query the
``/_matrix/key/v2/server/`` API on behalf of another server and sign the
response with their own key. A server may query multiple perspective servers
to ensure that they all report the same public keys.

Publishing Keys
_______________

Home servers publish the allowed TLS fingerprints and signing keys in a JSON
object at ``/_matrix/key/v2/server/${key_id}``. The response contains a list of
``verify_keys`` that are valid for signing federation requests made by the
server and for signing events. It contains a list of ``old_verify_keys``
which are only valid for signing events. Finally the response contains a list
of TLS certificate fingerprints to validate any connection made to the server.

A server may have multiple keys active at a given time. A server may have any
number of old keys. It is recommended that servers return a single JSON
response listing all of its keys whenever any ``key_id`` is requested to reduce
the number of round trips needed to discover the relevant keys for a server.
However a server may return a different responses for a different ``key_id``.

The ``tls_certificates`` contain a list of hashes of the X.509 TLS certificates
currently used by the server. The list must include SHA-256 hashes for every
certificate currently in use by the server. These fingerprints are valid until
the millisecond POSIX timestamp in ``valid_until_ts``.

The ``verify_keys`` can be used to sign requests and events made by the server
until the millisecond POSIX timestamp in ``valid_until_ts``. If a Home Server
receives an event with a ``origin_server_ts`` after the ``valid_until_ts`` then
it should request that ``key_id`` for the originating server to check whether
the key has expired.

The ``old_verify_keys`` can be used to sign events with an ``origin_server_ts``
before the ``expired_ts``. The ``expired_ts`` is a millisecond POSIX timestamp
of when the originating server stopped using that key.

Intermediate perspective servers should cache a response for half of its
remaining life time to avoid serving a stale response. Servers should avoid
querying for certificates more frequently than once an hour to avoid flooding
a server with requests.

==================== =================== ======================================
    Key                    Type                         Description
==================== =================== ======================================
``server_name``      String              DNS name of the home server.
``verify_keys``      Object              Public keys of the home server for
                                         verifying digital signatures.
``old_verify_keys``  Object              The public keys that the server used
                                         to use and when it stopped using them.
``signatures``       Object              Digital signatures for this object
                                         signed using the ``verify_keys``.
``tls_fingerprints`` Array of Objects    Hashes of X.509 TLS certificates used
                                         by this this server encoded as base64.
``valid_until_ts``   Integer             POSIX timestamp when the list of valid
                                         keys should be refreshed.
==================== =================== ======================================


.. code:: json

    {
        "old_verify_keys": {
{'fraction_non_alphanumeric': 0.0885582989329387, 'fraction_numerical': 0.0037386089259288107, 'mean_word_length': 3.409340659340659, 'pattern_counts': {'":': 4, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3297537', 'n_tokens_mistral': 3057, 'n_tokens_neox': 2890, 'n_words': 1742}