<commit_before>h1. ZooKeeper Plugin

h2. Usage

* Download ZooKeeper 3.3.3 from http://zookeeper.apache.org/releases.html
* Unzip the ZooKeeper archive into a directory, rename conf/zoo-sample.cnf into zoo.cnf and modify dataDir= line to point to a directory on your machine. Start ZooKeeper by running bin/zkServer.sh start.
* Install ZooKeeper plugin to ES
* Assuming that you are running ZooKeeper on the port 2181 (default), add the following lines to config/elasticsearch.yml file

<pre>
discovery:
    type: com.sonian.elasticsearch.zookeeper.discovery.ZooKeeperDiscoveryModule
sonian.elasticsearch.zookeeper:
    settings.enabled: true
    client.host: localhost:2181
    discovery.state_publishing.enabled: true
</pre>

* Start ES


h2. Operation

h3. Introduction

ES ZooKeeper Discovery plug-in is using ZooKeeper as an atomic, reliable, distributed database for node fault detection, master election and sharing common settings. It uses the fact that ZooKeeper can maintain a tree of nodes in which each node can have an associated value as well as child node, these nodes can be created and read atomically and clients can be notified about updates.

The ZooKeeper discovery process can be divided in 3 logical steps.

h3. Initialization and Registration

Upon ES node start, the ZooKeeper discovery plugin connects to ZooKeeper and creates the following persistent node if it doesn't already exist:<commit_msg>[eng-296] Add information about ZooKeeper-based settings to the plugin documentation<commit_after>h1. ZooKeeper Plugin

h2. Usage

h3. Installation

* Download ZooKeeper 3.3.3 from http://zookeeper.apache.org/releases.html
* Unzip the ZooKeeper archive into a directory, rename conf/zoo-sample.cnf into zoo.cnf and modify dataDir= line to point to a directory on your machine. Start ZooKeeper by running bin/zkServer.sh start.
* Install ZooKeeper plugin to ES
* Assuming that you are running ZooKeeper on the port 2181 (default), add the following lines to config/elasticsearch.yml file

<pre>
discovery:
    type: com.sonian.elasticsearch.zookeeper.discovery.ZooKeeperDiscoveryModule
sonian.elasticsearch.zookeeper:
    settings.enabled: true
    client.host: localhost:2181
    discovery.state_publishing.enabled: true
</pre>

* Start ES

The only mandatory parameter in the ZooKeeper plugin configuration is *client.host*. This parameter should contain an address of ZooKeeper server or comma-separate list of addresses if multiple servers are used.


h3. ZooKeeper state publishing

The *sonian.elasticsearch.zookeeper.discovery.state_publishing.enabled* parameter specifies if the plugin should use ZooKeeper to publish cluster state. By default, the cluster state is published using *org.elasticsearch.discovery.zen.publish.PublishClusterStateAction*, which causes master to send cluster state to all other nodes. If ZooKeeper-based state publishing is enabled, state is published inside ZooKeeper and all nodes are reading state from the ZooKeeper as soon as it changes.

h3. Settings

The *sonian.elasticsearch.zookeeper.settings.enabled* parameter specifies if the plugin should attempt to load configuration form ZooKeeper.

ZooKeeper Discovery plug-in can load settings from zookeeper. The *bin/zookeeper* utility can be used to upload new settings to ZooKeeper. The settings can be upload on per cluster basis as well as globally.

Assuming that current directory is ES_HOME, to upload cluster settings use:

<pre>
$ sh plugins/zookeeper/bin/zookeeper set elasticsearch.json
</pre>

To upload global settings use:

<pre>
$ sh plugins/zookeeper/bin/zookeeper set-global elasticsearch.json
</pre>

To retrieve current global or cluster settings use one of the following commands:

<pre>
$ sh plugins/zookeeper/bin/zookeeper get
$ sh plugins/zookeeper/bin/zookeeper get-global
</pre>

The settings are loaded in the following order:
* Local settings
* Global settings
* Cluster settings

The later loaded settings take precedence. In other words, cluster settings override global settings and global settings override local settings.

h2. Operation

h3. Introduction

ES ZooKeeper Discovery plug-in is using ZooKeeper as an atomic, reliable, distributed database for node fault detection, master election and sharing common settings. It uses the fact that ZooKeeper can maintain a tree of nodes in which each node can have an associated value as well as child node, these nodes can be created and read atomically and clients can be notified about updates.

The ZooKeeper discovery process can be divided in 3 logical steps.

h3. Initialization and Registration

Upon ES node start, the ZooKeeper discovery plugin connects to ZooKeeper and creates the following persistent node if it doesn't already exist:
{'fraction_non_alphanumeric': 0.049820561536837664, 'fraction_numerical': 0.008444162972345366, 'mean_word_length': 4.813496932515338, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1013409', 'n_tokens_mistral': 1343, 'n_tokens_neox': 1217, 'n_words': 597}