<commit_before>.. _timelock-migration:

Migration to External Timelock Services
=======================================

.. danger::

   Improperly configuring one's cluster to use external timestamp and lock services can result in **SEVERE DATA
   CORRUPTION**! Please contact the AtlasDB team before attempting a migration.

Why Migration?
--------------

AtlasDB assumes that timestamps returned by the timestamp service are monotonically increasing. In order to preserve
this guarantee when moving from an embedded timestamp service to an external timestamp service, we need to ensure
that timestamps issued by the external timestamp service are larger than those issued by the embedded one.
Otherwise, this can lead to serious data corruption.

The migration process must be run offline (that is, with no AtlasDB clients running during migration) and basically
consists of the following steps:

#. Set up the external timestamp service.
#. Obtain a timestamp ``TS`` that is at least as large as the largest timestamp issued.
#. Fast-forward the external timestamp service to at least ``TS``.
#. Invalidate the old timestamp service, preventing AtlasDB clients from retrieving timestamps from it.
#. Configure AtlasDB clients to retrieve timestamps from the new timestamp service.
#. Start AtlasDB clients.

Strictly speaking, step 4 is not required; that said, we prefer to do this to avoid the risk of data corruption in the
event of misconfiguration (that is, if some but not all clients are pointed to the new timestamp service).

Manual Migration
----------------

.. danger::

   Ensure that no AtlasDB clients are running during the migration process. Failure to ensure this can result in
   **SEVERE DATA CORRUPTION** as we are unable to guarantee that timestamps are monotonically increasing!

.. contents::
   :local:

Step 1: Setting up the Timelock Server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, setup the Timelock Server following the instructions in :ref:`timelock-installation`.

Step 2: Determining the AtlasDB Timestamp<commit_msg>Add step 0: shutdown your AtlasDB clients<commit_after>.. _timelock-migration:

Migration to External Timelock Services
=======================================

.. danger::

   Improperly configuring one's cluster to use external timestamp and lock services can result in **SEVERE DATA
   CORRUPTION**! Please contact the AtlasDB team before attempting a migration.

Why Migration?
--------------

AtlasDB assumes that timestamps returned by the timestamp service are monotonically increasing. In order to preserve
this guarantee when moving from an embedded timestamp service to an external timestamp service, we need to ensure
that timestamps issued by the external timestamp service are larger than those issued by the embedded one.
Otherwise, this can lead to serious data corruption.

The migration process must be run offline (that is, with no AtlasDB clients running during migration) and basically
consists of the following steps:

#. Set up the external timestamp service.
#. Obtain a timestamp ``TS`` that is at least as large as the largest timestamp issued.
#. Fast-forward the external timestamp service to at least ``TS``.
#. Invalidate the old timestamp service, preventing AtlasDB clients from retrieving timestamps from it.
#. Configure AtlasDB clients to retrieve timestamps from the new timestamp service.
#. Start AtlasDB clients.

Strictly speaking, step 4 is not required; that said, we prefer to do this to avoid the risk of data corruption in the
event of misconfiguration (that is, if some but not all clients are pointed to the new timestamp service).

Manual Migration
----------------

.. danger::

   Ensure that no AtlasDB clients are running during the migration process. Failure to ensure this can result in
   **SEVERE DATA CORRUPTION** as we are unable to guarantee that timestamps are monotonically increasing!

.. contents::
   :local:

Step 0: Shutdown your AtlasDB Clients
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Make sure that all of your AtlasDB clients are shut down. Failure to do this can result in severe data corruption,
because we cannot guarantee that the Timelock Server and the embedded AtlasDB timestamp services will together issue
monotonically increasing timestamps. (In fact, they almost certainly will not!)

Step 1: Setting up the Timelock Server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, setup the Timelock Server following the instructions in :ref:`timelock-installation`.

Step 2: Determining the AtlasDB Timestamp
{'fraction_non_alphanumeric': 0.09611992945326278, 'fraction_numerical': 0.001763668430335097, 'mean_word_length': 4.519464720194647, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3220917', 'n_tokens_mistral': 1098, 'n_tokens_neox': 1021, 'n_words': 612}