<commit_before>.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
.. use this file except in compliance with the License. You may obtain a copy of
.. the License at
..
..   http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
.. License for the specific language governing permissions and limitations under
.. the License.

.. _replication/protocol:

============================
CouchDB Replication Protocol
============================

:Version: 3

The `CouchDB Replication Protocol` is a protocol for synchronising JSON
documents between 2 peers over HTTP/1.1 by using the public :ref:`CouchDB REST
API <api>` and is based on the Apache CouchDB MVCC_ Data model.

Preface
=======

Language
--------

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in :rfc:`2119`.

Goals
-----

The primary goal of this specification is to describe the `CouchDB Replication
Protocol` under the hood.

The secondary goal is to provide enough detailed information about the protocol
to make it easy to build tools on any language and platform that can synchronize
data with CouchDB.

Definitions
-----------

JSON:
    :abbr:`JSON (JavaScript Object Notation)` is a text format for the
    serialization of structured data. It is described in `ECMA-262`_ and
    :rfc:`4627`.

URI:
    A URI is defined by :rfc:`3986`. It can be a URL as defined
    in :rfc:`1738`.

ID:
    An identifier (could be a UUID) as described in :rfc:`4122`.

Revision:
    A `MVCC`_ token value of following pattern: ``N-sig`` where ``N`` is ALWAYS
    a positive integer and ``sig`` is the Document signature (custom).
    Don't mix it up with the revision in version control systems!

Leaf Revision:
    The last Document Revision in a series of changes. Documents may have
    multiple Leaf Revisions (aka Conflict Revisions) due to concurrent updates.

Document:
    A document is a JSON object with an ID and Revision defined in ``_id`` and
    ``_rev`` fields respectively. A Document's ID MUST be unique within
    the Database where it is stored.

Database:
    A collection of Documents with a unique URI.

Changes Feed:
    A stream of Document-changing events (create, update, delete) for
    the specified Database.

Sequence ID:
    An ID provided by the Changes Feed. It MUST be incremental,
    but MAY NOT always be an integer.

Source:
    Database from where the Documents are replicated.

Target:
    Database where the Documents are replicated to.

Replication:
    The one-way directed synchronization process of Source and Target endpoints.

Checkpoint:
    Intermediate Recorded Sequence ID used for Replication recovery.

Replicator:
    A service or an application which initiates and runs Replication.

Filter Function:
    A special function of any programming language that is used to filter
    Documents during Replication (see :ref:`filterfun`)

Filter Function Name:
    An ID of a Filter Function that may be used as a symbolic reference (aka
    callback function) to apply the related Filter Function to Replication.

Filtered Replication:
    Replication of Documents from Source to Target using a Filter Function.

Full Replication:
    Replication of all Documents from Source to Target.

Push Replication:
    Replication process where Source is a local endpoint and Target is remote.

Pull Replication:
    Replication process where Source is a remote endpoint and Target is local.

Continuous Replication:
    Replication that "never stops": after processing all events from the
    Changes Feed, the Replicator doesn't close the connection, but awaits new
    change events from the Source. The connection is kept alive by periodic
    heartbeats.

Replication Log:
    A special Document that holds Replication history (recorded Checkpoints
    and a few more statistics) between Source and Target.

Replication ID:
    A unique value that unambiguously identifies the Replication Log.

Replication Protocol Algorithm
==============================

The `CouchDB Replication Protocol` is not *magical*, but
an agreement on usage of the public :ref:`CouchDB HTTP REST API <api>` to
enable Documents to be replicated from Source to Target.

The reference implementation, written in Erlang_, is provided by the
couch_replicator_ module in Apache CouchDB.

It is RECOMMENDED that one follow this algorithm specification, use the same
HTTP endpoints, and run requests with the same parameters to provide a
completely compatible implementation. Custom Replicator implementations MAY use
different HTTP API endpoints and request parameters depending on their local
specifics and they MAY implement only part of the Replication Protocol to run
only Push or Pull Replication. However, while such solutions could also run the
Replication process, they loose compatibility with the CouchDB Replicator.

Verify Peers
------------

.. code-block:: text

    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
    ' Verify Peers:                                                             '
    '                                                                           '
    '                404 Not Found   +--------------------------------+         '
    '       +----------------------- |     Check Source Existence     |         '
    '       |                        +--------------------------------+         '
    '       |                        |          HEAD /source          |         '
    '       |                        +--------------------------------+         '
    '       |                          |                                        '
    '       |                          | 200 OK                                 '
    '       |                          v                                        '
    '       |                        +--------------------------------+         '
    '       |                        |     Check Target Existence     | ----+   '
    '       |                        +--------------------------------+     |   '
    '       |                        |         HEAD /target           |     |   '
    '       |                        +--------------------------------+     |   '
    '       |                          |                                    |   '
    '       |                          | 404 Not Found                      |   '
    '       v                          v                                    |   '
    '   +-------+    No              +--------------------------------+     |   '
    '   | Abort | <----------------- |         Create Target?         |     |   '
    '   +-------+                    +--------------------------------+     |   '
    '       ^                          |                                    |   '
    '       |                          | Yes                                |   '
    '       |                          v                                    |   '
    '       |        Failure         +--------------------------------+     |   '
    '       +----------------------- |          Create Target         |     |   '
    '                                +--------------------------------+     |   '
    '                                |           PUT /target          |     |   '
    '                                +--------------------------------+     |   '
    '                                  |                                    |   '
    '                                  | 201 Created                 200 OK |   '
    '                                  |                                    |   '
    + - - - - - - - - - - - - - - - -  | - - - - - - - - - - - - - - - - -  | - +
                                       |                                    |
    + - - - - - - - - - - - - - - - -  | - - - - - - - - - - - - - - - - -  | - +
    ' Get Peers Information:           |                                    |   '
    '                                  +------------------------------------+   '
    '                                  |                                        '
    '                                  v                                        '
    '                                +--------------------------------+         '
    '                                |     Get Source Information     |         '
    '                                +--------------------------------+         '
    '                                                                           '
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +

The Replicator MUST ensure that both Source and Target exist
by using :head:`/{db}` requests.

Check Source Existence
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        HEAD /source HTTP/1.1
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Cache-Control: must-revalidate
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:50:39 GMT
        Server: CouchDB (Erlang/OTP)

Check Target Existence
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        HEAD /target HTTP/1.1
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Cache-Control: must-revalidate
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:51:11 GMT
        Server: CouchDB (Erlang/OTP)

Create Target?
^^^^^^^^^^^^^^

In case of a non-existent Target, the Replicator MAY make a :put:`/{db}`
request to create the Target:

    **Request**:

    .. code-block:: http

        PUT /target HTTP/1.1
        Accept: application/json
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 201 Created
        Content-Length: 12
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:58:41 GMT
        Server: CouchDB (Erlang/OTP)

        {
            "ok": true
        }

However, the Replicator's PUT request MAY NOT succeeded due to insufficient
privileges (which are granted by the provided credential) and so receive a
:statuscode:`401` or a :statuscode:`403` error. Such errors SHOULD be expected
and well handled:

    .. code-block:: http

        HTTP/1.1 500 Internal Server Error
        Cache-Control: must-revalidate
        Content-Length: 108
        Content-Type: application/json
        Date: Fri, 09 May 2014 13:50:32 GMT
        Server: CouchDB (Erlang OTP)

        {
            "error": "unauthorized",
            "reason": "unauthorized to access or create database http://localhost:5984/target"
        }

Abort
^^^^^

In case of a non-existent Source or Target, Replication SHOULD be aborted with
an HTTP error response:

    .. code-block:: http

        HTTP/1.1 500 Internal Server Error
        Cache-Control: must-revalidate
        Content-Length: 56
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:55:29 GMT
        Server: CouchDB (Erlang OTP)

        {
            "error": "db_not_found",
            "reason": "could not open source"
        }

Get Peers Information
---------------------

.. code-block:: text

    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
    ' Verify Peers:                                                    '
    '                         +------------------------+               '
    '                         | Check Target Existence |               '
    '                         +------------------------+               '
    '                                     |                            '
    '                                     | 200 OK                     '
    '                                     |                            '
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
                                          |
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
    ' Get Peers Information:              |                            '
    '                                     v                            '
    '                         +------------------------+               '
    '                         | Get Source Information |               '
    '                         +------------------------+               '
    '                         |      GET /source       |               '
    '                         +------------------------+               '
    '                                     |                            '
    '                                     | 200 OK                     '
    '                                     v                            '
    '                         +------------------------+               '
    '                         | Get Target Information |               '
    '                         +------------------------+               '
    '                         |      GET /target       |               '
    '                         +------------------------+               '
    '                                     |                            '
    '                                     | 200 OK                     '
    '                                     |                            '
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
                                          |
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
    ' Find Common Ancestry:               |                            '
    '                                     |                            '
    '                                     v                            '
    '                         +-------------------------+              '
    '                         | Generate Replication ID |              '
    '                         +-------------------------+              '
    '                                                                  '
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+

The Replicator retrieves basic information both from Source and Target using
:get:`/{db}` requests. The GET response MUST contain JSON objects with
the following mandatory fields:

- **instance_start_time** (*string*): Always ``"0"``. (Returned for legacy
  reasons.)
- **update_seq** (*number* / *string*): The current database Sequence ID.

Any other fields are optional. The information that the Replicator needs
is the ``update_seq`` field: this value will be used to define a *temporary*
(because Database data is subject to change) upper bound for changes feed
listening and statistic calculating to show proper Replication progress.

Get Source Information
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        GET /source HTTP/1.1
        Accept: application/json
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Cache-Control: must-revalidate
        Content-Length: 256
        Content-Type: application/json
        Date: Tue, 08 Oct 2013 07:53:08 GMT
        Server: CouchDB (Erlang OTP)

        {
            "committed_update_seq": 61772,
            "compact_running": false,
            "data_size": 70781613961,
            "db_name": "source",
            "disk_format_version": 6,
            "disk_size": 79132913799,
            "doc_count": 41961,
            "doc_del_count": 3807,
            "instance_start_time": "0",
            "purge_seq": 0,
            "update_seq": 61772
        }

Get Target Information
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        GET /target/ HTTP/1.1
        Accept: application/json
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Content-Length: 363
        Content-Type: application/json
        Date: Tue, 08 Oct 2013 12:37:01 GMT
        Server: CouchDB (Erlang/OTP)

        {
            "compact_running": false,
            "db_name": "target",
            "disk_format_version": 5,
            "disk_size": 77001455,
            "doc_count": 1832,
            "doc_del_count": 1,
            "instance_start_time": "0",
            "other": {
                "data_size": 50829452
            },
            "purge_seq": 0,
            "update_seq": "1841-g1AAAADveJzLYWBgYMlgTmGQT0lKzi9KdUhJMtbLSs1LLUst0k"
        }

Find Common Ancestry
------------------------

.. code-block:: text

    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
    ' Get Peers Information:                                                    '
    '                                                                           '
    '                             +-------------------------------------------+ '
    '                             |           Get Target Information          | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
                                    |
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
    ' Find Common Ancestry:         v                                           '
    '                             +-------------------------------------------+ '
    '                             |          Generate Replication ID          | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               |                                           '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |      Get Replication Log from Source      | '
    '                             +-------------------------------------------+ '
    '                             |     GET /source/_local/replication-id     | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               | 200 OK                                    '
    '                               | 404 Not Found                             '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |      Get Replication Log from Target      | '
    '                             +-------------------------------------------+ '
    '                             |     GET /target/_local/replication-id     | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               | 200 OK                                    '
    '                               | 404 Not Found                             '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |          Compare Replication Logs         | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               | Use latest common sequence as start point '
    '                               |                                           '
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
                                    |
                                    |
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
    ' Locate Changed Documents:     |                                           '
    '                               |                                           '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |        Listen Source Changes Feed         | '
    '                             +-------------------------------------------+ '
    '                                                                           '
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +

Generate Replication ID
^^^^^^^^^^^^^^^^^^^^^^^

Before Replication is started, the Replicator MUST generate a Replication ID.
This value is used to track Replication History, resume and continue previously
interrupted Replication process.

The Replication ID generation algorithm is implementation specific. Whatever
algorithm is used it MUST uniquely identify the Replication process. CouchDB's
Replicator, for example, uses the following factors in generating a Replication
ID:

- Persistent Peer UUID value. For CouchDB, the local
  :config:option:`Server UUID <couchdb/uuid>` is used
- Source and Target URI and if Source or Target are local or remote Databases
- If Target needed to be created
- If Replication is Continuous
- Any custom headers
- :ref:`Filter function <filterfun>` code if used
- Changes Feed query parameters, if any

.. note::
    See `couch_replicator_utils.erl`_ for an example of a Replication ID generation
    implementation.

    .. _couch_replicator_utils.erl: https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=blob;f=src/couch_replicator/src/couch_replicator_utils.erl;h=d7778db;hb=HEAD

Retrieve Replication Logs from Source and Target
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Once the Replication ID has been generated, the Replicator SHOULD retrieve
the Replication Log from both Source and Target using
:get:`/{db}/_local/{docid}`:

    **Request**:

    .. code-block:: http

        GET /source/_local/b3e44b920ee2951cb2e123b63044427a HTTP/1.1
        Accept: application/json<commit_msg>Fix broken link in Generate Replication ID section (#222)<commit_after>.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
.. use this file except in compliance with the License. You may obtain a copy of
.. the License at
..
..   http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
.. License for the specific language governing permissions and limitations under
.. the License.

.. _replication/protocol:

============================
CouchDB Replication Protocol
============================

:Version: 3

The `CouchDB Replication Protocol` is a protocol for synchronising JSON
documents between 2 peers over HTTP/1.1 by using the public :ref:`CouchDB REST
API <api>` and is based on the Apache CouchDB MVCC_ Data model.

Preface
=======

Language
--------

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in :rfc:`2119`.

Goals
-----

The primary goal of this specification is to describe the `CouchDB Replication
Protocol` under the hood.

The secondary goal is to provide enough detailed information about the protocol
to make it easy to build tools on any language and platform that can synchronize
data with CouchDB.

Definitions
-----------

JSON:
    :abbr:`JSON (JavaScript Object Notation)` is a text format for the
    serialization of structured data. It is described in `ECMA-262`_ and
    :rfc:`4627`.

URI:
    A URI is defined by :rfc:`3986`. It can be a URL as defined
    in :rfc:`1738`.

ID:
    An identifier (could be a UUID) as described in :rfc:`4122`.

Revision:
    A `MVCC`_ token value of following pattern: ``N-sig`` where ``N`` is ALWAYS
    a positive integer and ``sig`` is the Document signature (custom).
    Don't mix it up with the revision in version control systems!

Leaf Revision:
    The last Document Revision in a series of changes. Documents may have
    multiple Leaf Revisions (aka Conflict Revisions) due to concurrent updates.

Document:
    A document is a JSON object with an ID and Revision defined in ``_id`` and
    ``_rev`` fields respectively. A Document's ID MUST be unique within
    the Database where it is stored.

Database:
    A collection of Documents with a unique URI.

Changes Feed:
    A stream of Document-changing events (create, update, delete) for
    the specified Database.

Sequence ID:
    An ID provided by the Changes Feed. It MUST be incremental,
    but MAY NOT always be an integer.

Source:
    Database from where the Documents are replicated.

Target:
    Database where the Documents are replicated to.

Replication:
    The one-way directed synchronization process of Source and Target endpoints.

Checkpoint:
    Intermediate Recorded Sequence ID used for Replication recovery.

Replicator:
    A service or an application which initiates and runs Replication.

Filter Function:
    A special function of any programming language that is used to filter
    Documents during Replication (see :ref:`filterfun`)

Filter Function Name:
    An ID of a Filter Function that may be used as a symbolic reference (aka
    callback function) to apply the related Filter Function to Replication.

Filtered Replication:
    Replication of Documents from Source to Target using a Filter Function.

Full Replication:
    Replication of all Documents from Source to Target.

Push Replication:
    Replication process where Source is a local endpoint and Target is remote.

Pull Replication:
    Replication process where Source is a remote endpoint and Target is local.

Continuous Replication:
    Replication that "never stops": after processing all events from the
    Changes Feed, the Replicator doesn't close the connection, but awaits new
    change events from the Source. The connection is kept alive by periodic
    heartbeats.

Replication Log:
    A special Document that holds Replication history (recorded Checkpoints
    and a few more statistics) between Source and Target.

Replication ID:
    A unique value that unambiguously identifies the Replication Log.

Replication Protocol Algorithm
==============================

The `CouchDB Replication Protocol` is not *magical*, but
an agreement on usage of the public :ref:`CouchDB HTTP REST API <api>` to
enable Documents to be replicated from Source to Target.

The reference implementation, written in Erlang_, is provided by the
couch_replicator_ module in Apache CouchDB.

It is RECOMMENDED that one follow this algorithm specification, use the same
HTTP endpoints, and run requests with the same parameters to provide a
completely compatible implementation. Custom Replicator implementations MAY use
different HTTP API endpoints and request parameters depending on their local
specifics and they MAY implement only part of the Replication Protocol to run
only Push or Pull Replication. However, while such solutions could also run the
Replication process, they loose compatibility with the CouchDB Replicator.

Verify Peers
------------

.. code-block:: text

    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
    ' Verify Peers:                                                             '
    '                                                                           '
    '                404 Not Found   +--------------------------------+         '
    '       +----------------------- |     Check Source Existence     |         '
    '       |                        +--------------------------------+         '
    '       |                        |          HEAD /source          |         '
    '       |                        +--------------------------------+         '
    '       |                          |                                        '
    '       |                          | 200 OK                                 '
    '       |                          v                                        '
    '       |                        +--------------------------------+         '
    '       |                        |     Check Target Existence     | ----+   '
    '       |                        +--------------------------------+     |   '
    '       |                        |         HEAD /target           |     |   '
    '       |                        +--------------------------------+     |   '
    '       |                          |                                    |   '
    '       |                          | 404 Not Found                      |   '
    '       v                          v                                    |   '
    '   +-------+    No              +--------------------------------+     |   '
    '   | Abort | <----------------- |         Create Target?         |     |   '
    '   +-------+                    +--------------------------------+     |   '
    '       ^                          |                                    |   '
    '       |                          | Yes                                |   '
    '       |                          v                                    |   '
    '       |        Failure         +--------------------------------+     |   '
    '       +----------------------- |          Create Target         |     |   '
    '                                +--------------------------------+     |   '
    '                                |           PUT /target          |     |   '
    '                                +--------------------------------+     |   '
    '                                  |                                    |   '
    '                                  | 201 Created                 200 OK |   '
    '                                  |                                    |   '
    + - - - - - - - - - - - - - - - -  | - - - - - - - - - - - - - - - - -  | - +
                                       |                                    |
    + - - - - - - - - - - - - - - - -  | - - - - - - - - - - - - - - - - -  | - +
    ' Get Peers Information:           |                                    |   '
    '                                  +------------------------------------+   '
    '                                  |                                        '
    '                                  v                                        '
    '                                +--------------------------------+         '
    '                                |     Get Source Information     |         '
    '                                +--------------------------------+         '
    '                                                                           '
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +

The Replicator MUST ensure that both Source and Target exist
by using :head:`/{db}` requests.

Check Source Existence
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        HEAD /source HTTP/1.1
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Cache-Control: must-revalidate
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:50:39 GMT
        Server: CouchDB (Erlang/OTP)

Check Target Existence
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        HEAD /target HTTP/1.1
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Cache-Control: must-revalidate
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:51:11 GMT
        Server: CouchDB (Erlang/OTP)

Create Target?
^^^^^^^^^^^^^^

In case of a non-existent Target, the Replicator MAY make a :put:`/{db}`
request to create the Target:

    **Request**:

    .. code-block:: http

        PUT /target HTTP/1.1
        Accept: application/json
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 201 Created
        Content-Length: 12
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:58:41 GMT
        Server: CouchDB (Erlang/OTP)

        {
            "ok": true
        }

However, the Replicator's PUT request MAY NOT succeeded due to insufficient
privileges (which are granted by the provided credential) and so receive a
:statuscode:`401` or a :statuscode:`403` error. Such errors SHOULD be expected
and well handled:

    .. code-block:: http

        HTTP/1.1 500 Internal Server Error
        Cache-Control: must-revalidate
        Content-Length: 108
        Content-Type: application/json
        Date: Fri, 09 May 2014 13:50:32 GMT
        Server: CouchDB (Erlang OTP)

        {
            "error": "unauthorized",
            "reason": "unauthorized to access or create database http://localhost:5984/target"
        }

Abort
^^^^^

In case of a non-existent Source or Target, Replication SHOULD be aborted with
an HTTP error response:

    .. code-block:: http

        HTTP/1.1 500 Internal Server Error
        Cache-Control: must-revalidate
        Content-Length: 56
        Content-Type: application/json
        Date: Sat, 05 Oct 2013 08:55:29 GMT
        Server: CouchDB (Erlang OTP)

        {
            "error": "db_not_found",
            "reason": "could not open source"
        }

Get Peers Information
---------------------

.. code-block:: text

    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
    ' Verify Peers:                                                    '
    '                         +------------------------+               '
    '                         | Check Target Existence |               '
    '                         +------------------------+               '
    '                                     |                            '
    '                                     | 200 OK                     '
    '                                     |                            '
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
                                          |
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
    ' Get Peers Information:              |                            '
    '                                     v                            '
    '                         +------------------------+               '
    '                         | Get Source Information |               '
    '                         +------------------------+               '
    '                         |      GET /source       |               '
    '                         +------------------------+               '
    '                                     |                            '
    '                                     | 200 OK                     '
    '                                     v                            '
    '                         +------------------------+               '
    '                         | Get Target Information |               '
    '                         +------------------------+               '
    '                         |      GET /target       |               '
    '                         +------------------------+               '
    '                                     |                            '
    '                                     | 200 OK                     '
    '                                     |                            '
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
                                          |
    + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
    ' Find Common Ancestry:               |                            '
    '                                     |                            '
    '                                     v                            '
    '                         +-------------------------+              '
    '                         | Generate Replication ID |              '
    '                         +-------------------------+              '
    '                                                                  '
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+

The Replicator retrieves basic information both from Source and Target using
:get:`/{db}` requests. The GET response MUST contain JSON objects with
the following mandatory fields:

- **instance_start_time** (*string*): Always ``"0"``. (Returned for legacy
  reasons.)
- **update_seq** (*number* / *string*): The current database Sequence ID.

Any other fields are optional. The information that the Replicator needs
is the ``update_seq`` field: this value will be used to define a *temporary*
(because Database data is subject to change) upper bound for changes feed
listening and statistic calculating to show proper Replication progress.

Get Source Information
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        GET /source HTTP/1.1
        Accept: application/json
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Cache-Control: must-revalidate
        Content-Length: 256
        Content-Type: application/json
        Date: Tue, 08 Oct 2013 07:53:08 GMT
        Server: CouchDB (Erlang OTP)

        {
            "committed_update_seq": 61772,
            "compact_running": false,
            "data_size": 70781613961,
            "db_name": "source",
            "disk_format_version": 6,
            "disk_size": 79132913799,
            "doc_count": 41961,
            "doc_del_count": 3807,
            "instance_start_time": "0",
            "purge_seq": 0,
            "update_seq": 61772
        }

Get Target Information
^^^^^^^^^^^^^^^^^^^^^^

    **Request**:

    .. code-block:: http

        GET /target/ HTTP/1.1
        Accept: application/json
        Host: localhost:5984
        User-Agent: CouchDB

    **Response**:

    .. code-block:: http

        HTTP/1.1 200 OK
        Content-Length: 363
        Content-Type: application/json
        Date: Tue, 08 Oct 2013 12:37:01 GMT
        Server: CouchDB (Erlang/OTP)

        {
            "compact_running": false,
            "db_name": "target",
            "disk_format_version": 5,
            "disk_size": 77001455,
            "doc_count": 1832,
            "doc_del_count": 1,
            "instance_start_time": "0",
            "other": {
                "data_size": 50829452
            },
            "purge_seq": 0,
            "update_seq": "1841-g1AAAADveJzLYWBgYMlgTmGQT0lKzi9KdUhJMtbLSs1LLUst0k"
        }

Find Common Ancestry
------------------------

.. code-block:: text

    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
    ' Get Peers Information:                                                    '
    '                                                                           '
    '                             +-------------------------------------------+ '
    '                             |           Get Target Information          | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
                                    |
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
    ' Find Common Ancestry:         v                                           '
    '                             +-------------------------------------------+ '
    '                             |          Generate Replication ID          | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               |                                           '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |      Get Replication Log from Source      | '
    '                             +-------------------------------------------+ '
    '                             |     GET /source/_local/replication-id     | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               | 200 OK                                    '
    '                               | 404 Not Found                             '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |      Get Replication Log from Target      | '
    '                             +-------------------------------------------+ '
    '                             |     GET /target/_local/replication-id     | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               | 200 OK                                    '
    '                               | 404 Not Found                             '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |          Compare Replication Logs         | '
    '                             +-------------------------------------------+ '
    '                               |                                           '
    '                               | Use latest common sequence as start point '
    '                               |                                           '
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
                                    |
                                    |
    + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
    ' Locate Changed Documents:     |                                           '
    '                               |                                           '
    '                               v                                           '
    '                             +-------------------------------------------+ '
    '                             |        Listen Source Changes Feed         | '
    '                             +-------------------------------------------+ '
    '                                                                           '
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +

Generate Replication ID
^^^^^^^^^^^^^^^^^^^^^^^

Before Replication is started, the Replicator MUST generate a Replication ID.
This value is used to track Replication History, resume and continue previously
interrupted Replication process.

The Replication ID generation algorithm is implementation specific. Whatever
algorithm is used it MUST uniquely identify the Replication process. CouchDB's
Replicator, for example, uses the following factors in generating a Replication
ID:

- Persistent Peer UUID value. For CouchDB, the local
  :config:option:`Server UUID <couchdb/uuid>` is used
- Source and Target URI and if Source or Target are local or remote Databases
- If Target needed to be created
- If Replication is Continuous
- Any custom headers
- :ref:`Filter function <filterfun>` code if used
- Changes Feed query parameters, if any

.. note::
    See `couch_replicator_ids.erl`_ for an example of a Replication ID generation
    implementation.

    .. _couch_replicator_ids.erl: https://github.com/apache/couchdb/blob/master/src/couch_replicator/src/couch_replicator_ids.erl

Retrieve Replication Logs from Source and Target
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Once the Replication ID has been generated, the Replicator SHOULD retrieve
the Replication Log from both Source and Target using
:get:`/{db}/_local/{docid}`:

    **Request**:

    .. code-block:: http

        GET /source/_local/b3e44b920ee2951cb2e123b63044427a HTTP/1.1
        Accept: application/json
{'fraction_non_alphanumeric': 0.16101526868927227, 'fraction_numerical': 0.015180558309648137, 'mean_word_length': 0.802112284602557, 'pattern_counts': {'":': 56, '<': 13, '<?xml version=': 0, '>': 11, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 558, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4796381', 'n_tokens_mistral': 11166, 'n_tokens_neox': 10165, 'n_words': 2975}