<commit_before>---
title: Anti-entropy
aliases:
  - /enterprise_influxdb/v1.6/guides/anti-entropy/
menu:
  enterprise_influxdb_1_6:
    weight: 40
    parent: Administration
---

## Introduction

Shard entropy refers to inconsistency among shards in a shard group.
This can be due to the "eventually consistent" nature of data stored in InfluxDB
Enterprise clusters or due to missing or unreachable shards.
The anti-entropy (AE) service ensures that each data node has all the shards it
needs according to the metastore and that all shards in a shard group are consistent.
Missing shards are automatically repaired without operator intervention while
out-of-sync shards can be manually queued for repair.
This guide covers how AE works and some of the basic situations where it takes effect.

## Concepts

The anti-entropy service is part of the `influxd` process running on each data node
that ensures the node has all the shards the metastore says it should have and
that those shards are in sync with others in the same shard group.
If any shards are missing, the AE service will copy existing shards from other
shard owners.
If data inconsistencies are detected among shards in a shard group, you can
[envoke the AE process](#command-line-tools-for-managing-entropy) and queue the
out-of-sync shards for repair.
In the repair process, AE will sync the necessary updates from other shards
in the same shard group.

By default, the service checks every 30 seconds, as configured in the [`anti-entropy.check-interval`](/enterprise_influxdb/v1.6/administration/configuration/#check-interval-30s) setting.

The anti-entropy service can only address missing or inconsistent shards when
there is at least one copy of the shard available.
In other words, as long as new and healthy nodes are introduced, a replication
factor of 2 can recover from one missing or inconsistent node;
a replication factor of 3 can recover from two missing or inconsistent nodes, and so on.
A replication factor of 1 cannot be recovered by the anti-entropy service.

## Symptoms of entropy
The AE process automatically detects and fixes missing shards, but shard inconsistencies
must be [manually detected and queued for repair](#detecting-and-repairing-entropy).
There are symptoms of entropy that, if seen, would indicate an entropy repair is necessary.

### Different results for the same query
When running queries against an InfluxDB Enterprise cluster, each query may be routed to a different data node.
If entropy affects data within the queried range, the same query will return different
results depending on which node it is run against.

_**Query attempt 1**_
```
SELECT mean("usage_idle") WHERE time > '2018-06-06T18:00:00Z' AND time < '2018-06-06T18:15:00Z' GROUP BY time(3m) FILL(0)

name: cpu
time                  mean
----                  ----
1528308000000000000   99.11867392974537
1528308180000000000   99.15410822137049
1528308360000000000   99.14927494363032
1528308540000000000   99.1980535465783
1528308720000000000   99.18584290492262
```

_**Query attempt 2**_
```
SELECT mean("usage_idle") WHERE time > '2018-06-06T18:00:00Z' AND time < '2018-06-06T18:15:00Z' GROUP BY time(3m) FILL(0)

name: cpu
time                  mean
----                  ----
1528308000000000000   99.11867392974537
1528308180000000000   0
1528308360000000000   0
1528308540000000000   0
1528308720000000000   99.18584290492262
```

This indicates that data is missing in the queried time range and entropy is present.

### Flapping dashboards
A "flapping" dashboard means data visualizations changing when data is refreshed
and pulled from a node with entropy (inconsistent data).
It is the visual manifestation of getting [different results from the same query](#different-results-from-the-same-query).

<img src="/img/kapacitor/flapping-dashboard.gif" alt="Flapping dashboard" style="width:100%; max-width:800px">

## Technical details
<commit_msg>fixed broken link in ae 1.6 doc<commit_after>---
title: Anti-entropy
aliases:
  - /enterprise_influxdb/v1.6/guides/anti-entropy/
menu:
  enterprise_influxdb_1_6:
    weight: 40
    parent: Administration
---

## Introduction

Shard entropy refers to inconsistency among shards in a shard group.
This can be due to the "eventually consistent" nature of data stored in InfluxDB
Enterprise clusters or due to missing or unreachable shards.
The anti-entropy (AE) service ensures that each data node has all the shards it
needs according to the metastore and that all shards in a shard group are consistent.
Missing shards are automatically repaired without operator intervention while
out-of-sync shards can be manually queued for repair.
This guide covers how AE works and some of the basic situations where it takes effect.

## Concepts

The anti-entropy service is part of the `influxd` process running on each data node
that ensures the node has all the shards the metastore says it should have and
that those shards are in sync with others in the same shard group.
If any shards are missing, the AE service will copy existing shards from other
shard owners.
If data inconsistencies are detected among shards in a shard group, you can
[envoke the AE process](#command-line-tools-for-managing-entropy) and queue the
out-of-sync shards for repair.
In the repair process, AE will sync the necessary updates from other shards
in the same shard group.

By default, the service checks every 30 seconds, as configured in the [`anti-entropy.check-interval`](/enterprise_influxdb/v1.6/administration/configuration/#check-interval-30s) setting.

The anti-entropy service can only address missing or inconsistent shards when
there is at least one copy of the shard available.
In other words, as long as new and healthy nodes are introduced, a replication
factor of 2 can recover from one missing or inconsistent node;
a replication factor of 3 can recover from two missing or inconsistent nodes, and so on.
A replication factor of 1 cannot be recovered by the anti-entropy service.

## Symptoms of entropy
The AE process automatically detects and fixes missing shards, but shard inconsistencies
must be [manually detected and queued for repair](#detecting-and-repairing-entropy).
There are symptoms of entropy that, if seen, would indicate an entropy repair is necessary.

### Different results for the same query
When running queries against an InfluxDB Enterprise cluster, each query may be routed to a different data node.
If entropy affects data within the queried range, the same query will return different
results depending on which node it is run against.

_**Query attempt 1**_
```
SELECT mean("usage_idle") WHERE time > '2018-06-06T18:00:00Z' AND time < '2018-06-06T18:15:00Z' GROUP BY time(3m) FILL(0)

name: cpu
time                  mean
----                  ----
1528308000000000000   99.11867392974537
1528308180000000000   99.15410822137049
1528308360000000000   99.14927494363032
1528308540000000000   99.1980535465783
1528308720000000000   99.18584290492262
```

_**Query attempt 2**_
```
SELECT mean("usage_idle") WHERE time > '2018-06-06T18:00:00Z' AND time < '2018-06-06T18:15:00Z' GROUP BY time(3m) FILL(0)

name: cpu
time                  mean
----                  ----
1528308000000000000   99.11867392974537
1528308180000000000   0
1528308360000000000   0
1528308540000000000   0
1528308720000000000   99.18584290492262
```

This indicates that data is missing in the queried time range and entropy is present.

### Flapping dashboards
A "flapping" dashboard means data visualizations changing when data is refreshed
and pulled from a node with entropy (inconsistent data).
It is the visual manifestation of getting [different results from the same query](#different-results-for-the-same-query).

<img src="/img/kapacitor/flapping-dashboard.gif" alt="Flapping dashboard" style="width:100%; max-width:800px">

## Technical details

{'fraction_non_alphanumeric': 0.0626031746031746, 'fraction_numerical': 0.09853968253968254, 'mean_word_length': 4.307277628032345, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 20, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3632343', 'n_tokens_mistral': 2741, 'n_tokens_neox': 2161, 'n_words': 1004}