<commit_before>Introduction
============

This document describes a collection of device-mapper targets that
between them implement thin-provisioning and snapshots.

The main highlight of this implementation, compared to the previous
implementation of snapshots, is that it allows many virtual devices to
be stored on the same data volume.  This simplifies administration and
allows the sharing of data between volumes, thus reducing disk usage.

Another significant feature is support for an arbitrary depth of
recursive snapshots (snapshots of snapshots of snapshots ...).  The
previous implementation of snapshots did this by chaining together
lookup tables, and so performance was O(depth).  This new
implementation uses a single data structure to avoid this degradation
with depth.  Fragmentation may still be an issue, however, in some
scenarios.

Metadata is stored on a separate device from data, giving the
administrator some freedom, for example to:

- Improve metadata resilience by storing metadata on a mirrored volume
  but data on a non-mirrored one.

- Improve performance by storing the metadata on SSD.

Status
======

These targets are very much still in the EXPERIMENTAL state.  Please
do not yet rely on them in production.  But do experiment and offer us
feedback.  Different use cases will have different performance
characteristics, for example due to fragmentation of the data volume.

If you find this software is not performing as expected please mail
<EMAIL> with details and we'll try our best to improve
things for you.

Userspace tools for checking and repairing the metadata are under
development.

Cookbook
========

This section describes some quick recipes for using thin provisioning.
They use the dmsetup program to control the device-mapper driver
directly.  End users will be advised to use a higher-level volume
manager such as LVM2 once support has been added.

Pool device
-----------

The pool device ties together the metadata volume and the data volume.
It maps I/O linearly to the data volume and updates the metadata via
two mechanisms:

- Function calls from the thin targets

- Device-mapper 'messages' from userspace which control the creation of new
  virtual devices amongst other things.

Setting up a fresh pool device
------------------------------

Setting up a pool device requires a valid metadata device, and a
data device.  If you do not have an existing metadata device you can
make one by zeroing the first 4k to indicate empty metadata.

    dd if=/dev/zero of=$metadata_dev bs=4096 count=1

The amount of metadata you need will vary according to how many blocks
are shared between thin devices (i.e. through snapshots).  If you have
less sharing than average you'll need a larger-than-average metadata device.

As a guide, we suggest you calculate the number of bytes to use in the
metadata device as 48 * $data_dev_size / $data_block_size but round it up
to 2MB if the answer is smaller.  If you're creating large numbers of
snapshots which are recording large amounts of change, you may find you
need to increase this.

The largest size supported is 16GB: If the device is larger,
a warning will be issued and the excess space will not be used.

Reloading a pool table
----------------------

You may reload a pool's table, indeed this is how the pool is resized
if it runs out of space.  (N.B. While specifying a different metadata
device when reloading is not forbidden at the moment, things will go
wrong if it does not route I/O to exactly the same on-disk location as
previously.)

Using an existing pool device
-----------------------------

    dmsetup create pool \
	--table "0 20971520 thin-pool $metadata_dev $data_dev \
		 $data_block_size $low_water_mark"

$data_block_size gives the smallest unit of disk space that can be
allocated at a time expressed in units of 512-byte sectors.  People
primarily interested in thin provisioning may want to use a value such
as 1024 (512KB).  People doing lots of snapshotting may want a smaller value
such as 128 (64KB).  If you are not zeroing newly-allocated data,
a larger $data_block_size in the region of 256000 (128MB) is suggested.
$data_block_size must be the same for the lifetime of the
metadata device.

$low_water_mark is expressed in blocks of size $data_block_size.  If
free space on the data device drops below this level then a dm event
will be triggered which a userspace daemon should catch allowing it to
extend the pool device.  Only one such event will be sent.
Resuming a device with a new table itself triggers an event so the
userspace daemon can use this to detect a situation where a new table
already exceeds the threshold.

Thin provisioning
-----------------

i) Creating a new thinly-provisioned volume.

  To create a new thinly- provisioned volume you must send a message to an
  active pool device, /dev/mapper/pool in this example.

    dmsetup message /dev/mapper/pool 0 "create_thin 0"

  Here '0' is an identifier for the volume, a 24-bit number.  It's up<commit_msg>dm thin: add data block size limits to Documentation<commit_after>Introduction
============

This document describes a collection of device-mapper targets that
between them implement thin-provisioning and snapshots.

The main highlight of this implementation, compared to the previous
implementation of snapshots, is that it allows many virtual devices to
be stored on the same data volume.  This simplifies administration and
allows the sharing of data between volumes, thus reducing disk usage.

Another significant feature is support for an arbitrary depth of
recursive snapshots (snapshots of snapshots of snapshots ...).  The
previous implementation of snapshots did this by chaining together
lookup tables, and so performance was O(depth).  This new
implementation uses a single data structure to avoid this degradation
with depth.  Fragmentation may still be an issue, however, in some
scenarios.

Metadata is stored on a separate device from data, giving the
administrator some freedom, for example to:

- Improve metadata resilience by storing metadata on a mirrored volume
  but data on a non-mirrored one.

- Improve performance by storing the metadata on SSD.

Status
======

These targets are very much still in the EXPERIMENTAL state.  Please
do not yet rely on them in production.  But do experiment and offer us
feedback.  Different use cases will have different performance
characteristics, for example due to fragmentation of the data volume.

If you find this software is not performing as expected please mail
<EMAIL> with details and we'll try our best to improve
things for you.

Userspace tools for checking and repairing the metadata are under
development.

Cookbook
========

This section describes some quick recipes for using thin provisioning.
They use the dmsetup program to control the device-mapper driver
directly.  End users will be advised to use a higher-level volume
manager such as LVM2 once support has been added.

Pool device
-----------

The pool device ties together the metadata volume and the data volume.
It maps I/O linearly to the data volume and updates the metadata via
two mechanisms:

- Function calls from the thin targets

- Device-mapper 'messages' from userspace which control the creation of new
  virtual devices amongst other things.

Setting up a fresh pool device
------------------------------

Setting up a pool device requires a valid metadata device, and a
data device.  If you do not have an existing metadata device you can
make one by zeroing the first 4k to indicate empty metadata.

    dd if=/dev/zero of=$metadata_dev bs=4096 count=1

The amount of metadata you need will vary according to how many blocks
are shared between thin devices (i.e. through snapshots).  If you have
less sharing than average you'll need a larger-than-average metadata device.

As a guide, we suggest you calculate the number of bytes to use in the
metadata device as 48 * $data_dev_size / $data_block_size but round it up
to 2MB if the answer is smaller.  If you're creating large numbers of
snapshots which are recording large amounts of change, you may find you
need to increase this.

The largest size supported is 16GB: If the device is larger,
a warning will be issued and the excess space will not be used.

Reloading a pool table
----------------------

You may reload a pool's table, indeed this is how the pool is resized
if it runs out of space.  (N.B. While specifying a different metadata
device when reloading is not forbidden at the moment, things will go
wrong if it does not route I/O to exactly the same on-disk location as
previously.)

Using an existing pool device
-----------------------------

    dmsetup create pool \
	--table "0 20971520 thin-pool $metadata_dev $data_dev \
		 $data_block_size $low_water_mark"

$data_block_size gives the smallest unit of disk space that can be
allocated at a time expressed in units of 512-byte sectors.
$data_block_size must be between 128 (64KB) and 2097152 (1GB) and a
multiple of 128 (64KB).  $data_block_size cannot be changed after the
thin-pool is created.  People primarily interested in thin provisioning
may want to use a value such as 1024 (512KB).  People doing lots of
snapshotting may want a smaller value such as 128 (64KB).  If you are
not zeroing newly-allocated data, a larger $data_block_size in the
region of 256000 (128MB) is suggested.

$low_water_mark is expressed in blocks of size $data_block_size.  If
free space on the data device drops below this level then a dm event
will be triggered which a userspace daemon should catch allowing it to
extend the pool device.  Only one such event will be sent.
Resuming a device with a new table itself triggers an event so the
userspace daemon can use this to detect a situation where a new table
already exceeds the threshold.

Thin provisioning
-----------------

i) Creating a new thinly-provisioned volume.

  To create a new thinly- provisioned volume you must send a message to an
  active pool device, /dev/mapper/pool in this example.

    dmsetup message /dev/mapper/pool 0 "create_thin 0"

  Here '0' is an identifier for the volume, a 24-bit number.  It's up
{'fraction_non_alphanumeric': 0.05764312413928782, 'fraction_numerical': 0.011607318512689356, 'mean_word_length': 3.786723163841808, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7143461', 'n_tokens_mistral': 2680, 'n_tokens_neox': 2486, 'n_words': 1552}