<commit_before>Daedal
======

This repository contains a set of Ruby classes designed to make ElasticSearch
query creation simpler and easier to debug. The goal is to reproduce all
components of the ElasticSearch [Query DSL](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html)
to aid in the construction of complex queries. Type checking and attribute
coercion are handled using [Virtus](https://github.com/solnic/virtus) to make
it harder to construct invalid ElasticSearch queries before sending them to the server.

The ElasticSearch Query DSL is huge! There are also a ton of different options within each
component. My goal is to include as much of that functionality and flexibility as possible, but
also maintain as high of test coverage as possible. That means it'll take some time for this project
to reach full coverage of the Query DSL, so please feel free to contribute or be patient :)

Installation
------------

``` terminal
$ gem install daedal
```

or in your `Gemfile`

``` ruby
gem 'daedal'
```

Usage
--------

### ElasticSearch Query DSL

Other Ruby packages for ElasticSearch allow you to create queries either as
hashes or by constructing raw JSON:

``` ruby
match_query = {'match' => {'foo' => {'query' => 'bar'}}}
```

For more complicated queries, dealing with the resulting large nested hash can be
frustrating. Inspired by ElasticSearch's built in 
[Java API](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/),
Daedal contains Ruby classes designed to make query construction more Ruby-like.

### Queries

Queries are contained within the `Queries` module. You can construct query components like:

``` ruby
require 'daedal'

# creates the basic match query
match_query = Daedal::Queries::MatchQuery.new(field: 'foo', query: 'bar')
```
Each query object has `#to_json` defined for easy conversion for use with any of the Ruby
ElasticSearch clients out there:
``` ruby
match_query.to_json # => "{\"match\":{\"foo\":{\"query\":\"bar\"}}}"
```

To date (12/8/2013), I have implemented the following queries:
* [bool query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html)
* [constant score query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-constant-score-query.html)
* [dis max query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-dis-max-query.html)
* [filtered query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-filtered-query.html)
* [match all query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html)
* [match query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html)
* [multi match query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html)

### Filters

Filters are contained within the `Filters` module. You can construct filter components
in the same way as queries:

``` ruby
require 'daedal'

term_filter = Daedal::Filters::TermFilter.new(field: 'foo', term: 'bar')

term_filter.to_json # => "{\"term\":{\"foo\":\"bar\"}}"
```

To date (12/8/2013), I have implemented the following filters:
* [term filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html)

### Type checking and attribute coercion

When creating ElasticSearch queries via nested hashes, it is all too easy to
assign an invalid value to a specific field, which would then result in
an error response when sending the query to the server. For instance, the query:

``` ruby
constant_score_query = {'constant_score' => {'boost' => 'foo', 'query': {'match_all': {}}}}
```

would yield a server error, since the `boost` parameter must be a number.

Daedal uses [Virtus](https://github.com/solnic/virtus) to perform data-type coercions. 
That way, invalid query parameters are surfaced at runtime, making debugging easier.
The previous `constant_score_query` example in Daedal would raise an error:

``` ruby
match_all_query = Daedal::Queries::MatchAllQuery.new()
constant_score_query = Daedal::Queries::ConstantScoreQuery.new(boost: 'foo', query: match_all_query)

# Virtus::CoercionError: Failed to coerce "foo" into Float
```
<commit_msg>Updated readme with progress<commit_after>Daedal
======

This repository contains a set of Ruby classes designed to make ElasticSearch
query creation simpler and easier to debug. The goal is to reproduce all
components of the ElasticSearch [Query DSL](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html)
to aid in the construction of complex queries. Type checking and attribute
coercion are handled using [Virtus](https://github.com/solnic/virtus) to make
it harder to construct invalid ElasticSearch queries before sending them to the server.

The ElasticSearch Query DSL is huge! There are also a ton of different options within each
component. My goal is to include as much of that functionality and flexibility as possible, but
also maintain as high of test coverage as possible. That means it'll take some time for this project
to reach full coverage of the Query DSL, so please feel free to contribute or be patient :)

Installation
------------

``` terminal
$ gem install daedal
```

or in your `Gemfile`

``` ruby
gem 'daedal'
```

Usage
--------

### ElasticSearch Query DSL

Other Ruby packages for ElasticSearch allow you to create queries either as
hashes or by constructing raw JSON:

``` ruby
match_query = {'match' => {'foo' => {'query' => 'bar'}}}
```

For more complicated queries, dealing with the resulting large nested hash can be
frustrating. Inspired by ElasticSearch's built in 
[Java API](http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/),
Daedal contains Ruby classes designed to make query construction more Ruby-like.

### Queries

Queries are contained within the `Queries` module. You can construct query components like:

``` ruby
require 'daedal'

# creates the basic match query
match_query = Daedal::Queries::MatchQuery.new(field: 'foo', query: 'bar')
```
Each query object has `#to_json` defined for easy conversion for use with any of the Ruby
ElasticSearch clients out there:
``` ruby
match_query.to_json # => "{\"match\":{\"foo\":{\"query\":\"bar\"}}}"
```

To date (12/8/2013), I have implemented the following queries:
* [bool query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html)
* [constant score query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-constant-score-query.html)
* [dis max query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-dis-max-query.html)
* [filtered query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-filtered-query.html)
* [match all query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html)
* [match query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html)
* [multi match query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html)
* [nested query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-nested-query.html)
* [prefix query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-prefix-query.html)

To be implemented next:
* [function score query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html)
* [fuzzy query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-fuzzy-query.html)
* [query string query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html)
* [range query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-range-query.html)
* [regexp query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-regexp-query.html)
* [term query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-query.html)
* [terms query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html)
* [wildcard query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html)

Queries I'm not planning on implementing at all, since they're deprecated:
* [custom filters score query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-custom-filters-score-query.html)
* [custom score query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-custom-score-query.html)
* [custom boost factor query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-custom-boost-factor-query.html)
* [text query](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-text-query.html)


### Filters

Filters are contained within the `Filters` module. You can construct filter components
in the same way as queries:

``` ruby
require 'daedal'

term_filter = Daedal::Filters::TermFilter.new(field: 'foo', term: 'bar')

term_filter.to_json # => "{\"term\":{\"foo\":\"bar\"}}"
```

To date (12/8/2013), I have implemented the following filters:
* [and filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-and-filter.html)
* [geo distance filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-filter.html)
* [range filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-range-filter.html)
* [term filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html)
* [terms filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html)

To be implemented next:
* [bool filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-bool-filter.html)
* [nested filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-nested-filter.html)
* [or filter](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-or-filter.html)

### Type checking and attribute coercion

When creating ElasticSearch queries via nested hashes, it is all too easy to
assign an invalid value to a specific field, which would then result in
an error response when sending the query to the server. For instance, the query:

``` ruby
constant_score_query = {'constant_score' => {'boost' => 'foo', 'query': {'match_all': {}}}}
```

would yield a server error, since the `boost` parameter must be a number.

Daedal uses [Virtus](https://github.com/solnic/virtus) to perform data-type coercions. 
That way, invalid query parameters are surfaced at runtime, making debugging easier.
The previous `constant_score_query` example in Daedal would raise an error:

``` ruby
match_all_query = Daedal::Queries::MatchAllQuery.new()
constant_score_query = Daedal::Queries::ConstantScoreQuery.new(boost: 'foo', query: match_all_query)

# Virtus::CoercionError: Failed to coerce "foo" into Float
```

{'fraction_non_alphanumeric': 0.1322314049586777, 'fraction_numerical': 0.0024617548795498505, 'mean_word_length': 5.182065217391305, 'pattern_counts': {'":': 10, '<': 3, '<?xml version=': 0, '>': 17, 'https://': 4, 'lorem ipsum': 0, 'www.': 41, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2510701', 'n_tokens_mistral': 3405, 'n_tokens_neox': 3337, 'n_words': 930}