<commit_before>h1. Script Facet Plugin for ElasticSearch

h2. Introduction

p. The script facet plugin provides fully scriptable facets for elasticsearch.

h2. Compatibility

|_. Script Facet Plugin   |_.  Elasticsearch    |
| master        |  0.19.10 -> master   |
| 1.0.0         |  0.19.10 -> master   |

h2. Usage

p. In order to install the plugin, simply run: @bin/plugin -install imotov/elasticsearch-facet-script/1.0.0@

p. The script facet plugin can be used for quick custom facet prototyping. The script facet is using three script to initialize, collect and aggregate the facets. A typical script facet request looks like this:

<pre>
"facets": {
    "facet1": {
        "script": {
            "init_script" : "my_init",
            "map_script": "my_map",
            "combine_script": "my_combine",
            "reduce_script" : "my_reduce",
            "params" : {
                "facet" : [],
                "param1" : "value 1"
            }
        }
    }
}
</pre>

p. A script facet execution can be represented using the following pseudocode:

<pre>
facets = [];
foreach(shard in shards) {
    init_script(); // Executed once per shard
    foreach(record in search_results(shard)) {
        // Init _field and doc lookup from the record
        map_script(); // Executed once per record
    }
    facets.add(combine_script()); // Executed once per shard after all records are processed
}
reduce_script(facets); // Executed once per facet request
</pre>

p. The @init_script@, @map_script@ and @combine_script@ scripts are executed on the nodes where shards are allocated. The @reduce_script@ is executed on the node that received the client's request.

p. The @init_script@, @map_script@ and @combine_script@ scripts can access parameters specified in the @params@ field of the request. These scripts can also use node client using @_client@ variable and search context using @_ctx@ variable. The @map_script@ can  access the current record using standard "document, field and source lookup mechanism":http://www.elasticsearch.org/guide/reference/modules/scripting.html.

p. The return values of the @combine_script@ are accumulated into an array list that is later passed to the @reduce_script@ script as a @facets@ parameter. The return value of the @reduce_script@ are returned to the users as a result of the facet query. It's important to note that return values of the @combine_script@ and @reduce_script@ scripts have to be JSON serializable.

p. The only mandatory parameter of the script facet is @map_script@. By default, the @init_script@ doesn't do anything. the @combine_script@ returns the variable named @facet@ and @reduce_script@ simply returns the array of the facets that it received from the shards.

h2. Examples

p. The following request calculates letter frequencies for the letters 'A'-'Z' in the field @message@.
* "examples/charfreq.sh":https://github.com/imotov/elasticsearch-facet-script/blob/master/examples/charfreq.sh
* "config/scripts/charfreq_init.mvel":https://github.com/imotov/elasticsearch-facet-script/blob/master/config/scripts/charfreq_init.mvel
* "config/scripts/charfreq_map.mvel":https://github.com/imotov/elasticsearch-facet-script/blob/master/config/scripts/charfreq_map.mvel
* "config/scripts/charfreq_reduce.mvel":https://github.com/imotov/elasticsearch-facet-script/blob/master/config/scripts/charfreq_reduce.mvel<commit_msg>Add description of params handling to README<commit_after>h1. Script Facet Plugin for ElasticSearch

h2. Introduction

p. The script facet plugin provides fully scriptable facets for elasticsearch.

h2. Compatibility

|_. Script Facet Plugin   |_.  Elasticsearch    |
| master        |  0.19.10 -> master   |
| 1.0.0         |  0.19.10 -> master   |

h2. Usage

p. In order to install the plugin, simply run: @bin/plugin -install imotov/elasticsearch-facet-script/1.0.0@

p. The script facet plugin can be used for quick custom facet prototyping. The script facet is using three script to initialize, collect and aggregate the facets. A typical script facet request looks like this:

<pre>
"facets": {
    "facet1": {
        "script": {
            "init_script" : "my_init",
            "map_script": "my_map",
            "combine_script": "my_combine",
            "reduce_script" : "my_reduce",
            "params" : {
                "facet" : [],
                "param1" : "value 1"
            }
        }
    }
}
</pre>

p. A script facet execution can be represented using the following pseudocode:

<pre>
facets = [];
foreach(shard in shards) {
    init_script(); // Executed once per shard
    foreach(record in search_results(shard)) {
        // Init _field and doc lookup from the record
        map_script(); // Executed once per record
    }
    facets.add(combine_script()); // Executed once per shard after all records are processed
}
reduce_script(facets); // Executed once per facet request
</pre>

p. The @init_script@, @map_script@ and @combine_script@ scripts are executed on the nodes where shards are allocated. The @reduce_script@ is executed on the node that received the client's request.

p. The @init_script@, @map_script@ and @combine_script@ scripts can access parameters specified in the @params@ field of the request. These scripts can also use node client using @_client@ variable and search context using @_ctx@ variable. The @map_script@ can access the current record using standard "document, field and source lookup mechanism":http://www.elasticsearch.org/guide/reference/modules/scripting.html.

p. The content of the @params@ field is initialized with values specified in the facet requests at the beginning of a shard processing and then preserved between all script calls within the shard. The @init_script@ can be used to do additional initialization of the @params@ map, @map_script@ can use the @params@ map to accumulate results of processing and @combine_script@ can retrieve accumulated results from the @params@ map and combine them into intermediate facet for the processed shard. The return values of @combine_script@ calls for all shards are sent to the node where @reduce_script@ is running and accumulated into an array list that is passed to the @reduce_script@ script as a @facets@ parameter. The return value of the @reduce_script@ are returned to the users as a result of the facet query. It's important to note that return values of the @combine_script@ and @reduce_script@ scripts have to be JSON serializable, which means they can contain only primitive data types, @java.util.Date@, @byte[]@, @Object[]@, @java.util.List@, and @java.util.Map@.

p. The only mandatory parameter of the script facet is @map_script@. By default, the @init_script@ doesn't do anything, the @combine_script@ returns the variable named @facet@ and @reduce_script@ simply returns the array of the facets that it received from the shards.

h2. Examples

p. The following request calculates letter frequencies for the letters 'A'-'Z' in the field @message@.
* "examples/charfreq.sh":https://github.com/imotov/elasticsearch-facet-script/blob/master/examples/charfreq.sh
* "config/scripts/charfreq_init.mvel":https://github.com/imotov/elasticsearch-facet-script/blob/master/config/scripts/charfreq_init.mvel
* "config/scripts/charfreq_map.mvel":https://github.com/imotov/elasticsearch-facet-script/blob/master/config/scripts/charfreq_map.mvel
* "config/scripts/charfreq_reduce.mvel":https://github.com/imotov/elasticsearch-facet-script/blob/master/config/scripts/charfreq_reduce.mvel
{'fraction_non_alphanumeric': 0.0924325043223833, 'fraction_numerical': 0.00638382763665381, 'mean_word_length': 3.792861695347355, 'pattern_counts': {'":': 20, '<': 11, '<?xml version=': 0, '>': 15, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4904066', 'n_tokens_mistral': 2183, 'n_tokens_neox': 2124, 'n_words': 811}