<commit_before>#Jest

Jest is a Java HTTP Rest client for [ElasticSearch](http://www.elasticsearch.org).

ElasticSearch is an Open Source (Apache 2), Distributed, RESTful, Search Engine built on top of Apache Lucene.

ElasticSearch already has a Java API which is also used by ElasticSearch internally, [but Jest fills a gap, it is the missing client for ElasticSearch Http Rest interface](#comparison-to-native-api).

>Read great [introduction](http://www.ibm.com/developerworks/java/library/j-javadev2-24/index.html?ca=drs-) to ElasticSearch and Jest from IBM Developer works.

Installation
------------

### Stable

Jest maven repository is hosted on [Sonatype](http://www.sonatype.org) which is then synced to [Maven Central](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22io.searchbox%22) with a slight delay.

To get the latest *stable* version without waiting for Maven Central sync add Sonatype repository definition to your root pom.xml

``` xml
<repositories>
.
.
 <repository>
   <id>sonatype</id>
   <name>Sonatype Groups</name>
   <url>https://oss.sonatype.org/content/groups/public/</url>
 </repository>
.
.
</repositories>

```

Add Jest as a dependency to your project.


``` xml
<dependency>
  <groupId>io.searchbox</groupId>
  <artifactId>jest</artifactId>
  <version>2.0.0</version>
</dependency>
```

### Snapshot

Jest also publishes a snapshot version on [Sonatype Snapshot Repository](https://oss.sonatype.org/content/repositories/snapshots) after every push resulting in a successfull build.


To get the latest *snapshot* version add Sonatype Snapshot Repository definition to your root pom.xml

``` xml
<repositories>
.
.
  <repository>
    <id>snapshots-repo</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    <releases><enabled>false</enabled></releases>
    <snapshots><enabled>true</enabled></snapshots>
  </repository>
.
.
</repositories>

```

Add Jest snapshot as a dependency to your project.


``` xml
<dependency>
  <groupId>io.searchbox</groupId>
  <artifactId>jest</artifactId>
  <version>2.0.1-SNAPSHOT</version>
</dependency>
```

>Ensure to check [the Changelog](https://github.com/searchbox-io/Jest/wiki/Changelog).

Usage
------------

Start using Jest by simply creating a `JestClient` instance:

``` java
 // Construct a new Jest client according to configuration via factory
 JestClientFactory factory = new JestClientFactory();
 factory.setHttpClientConfig(new HttpClientConfig
                        .Builder("http://localhost:9200")
                        .multiThreaded(true)
                        .build());
 JestClient client = factory.getObject();
```
> `JestClient` is designed to be singleton, don't construct it for each request!

### Creating an Index<commit_msg>Clarifying the importance of setting conn limits<commit_after>#Jest

Jest is a Java HTTP Rest client for [ElasticSearch](http://www.elasticsearch.org).

ElasticSearch is an Open Source (Apache 2), Distributed, RESTful, Search Engine built on top of Apache Lucene.

ElasticSearch already has a Java API which is also used by ElasticSearch internally, [but Jest fills a gap, it is the missing client for ElasticSearch Http Rest interface](#comparison-to-native-api).

>Read great [introduction](http://www.ibm.com/developerworks/java/library/j-javadev2-24/index.html?ca=drs-) to ElasticSearch and Jest from IBM Developer works.

Installation
------------

### Stable

Jest maven repository is hosted on [Sonatype](http://www.sonatype.org) which is then synced to [Maven Central](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22io.searchbox%22) with a slight delay.

To get the latest *stable* version without waiting for Maven Central sync add Sonatype repository definition to your root pom.xml

``` xml
<repositories>
.
.
 <repository>
   <id>sonatype</id>
   <name>Sonatype Groups</name>
   <url>https://oss.sonatype.org/content/groups/public/</url>
 </repository>
.
.
</repositories>

```

Add Jest as a dependency to your project.


``` xml
<dependency>
  <groupId>io.searchbox</groupId>
  <artifactId>jest</artifactId>
  <version>2.0.0</version>
</dependency>
```

### Snapshot

Jest also publishes a snapshot version on [Sonatype Snapshot Repository](https://oss.sonatype.org/content/repositories/snapshots) after every push resulting in a successfull build.


To get the latest *snapshot* version add Sonatype Snapshot Repository definition to your root pom.xml

``` xml
<repositories>
.
.
  <repository>
    <id>snapshots-repo</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    <releases><enabled>false</enabled></releases>
    <snapshots><enabled>true</enabled></snapshots>
  </repository>
.
.
</repositories>

```

Add Jest snapshot as a dependency to your project.


``` xml
<dependency>
  <groupId>io.searchbox</groupId>
  <artifactId>jest</artifactId>
  <version>2.0.1-SNAPSHOT</version>
</dependency>
```

>Ensure to check [the Changelog](https://github.com/searchbox-io/Jest/wiki/Changelog).

Usage
------------

Start using Jest by simply creating a `JestClient` instance:

``` java
 // Construct a new Jest client according to configuration via factory
 JestClientFactory factory = new JestClientFactory();
 factory.setHttpClientConfig(new HttpClientConfig
                        .Builder("http://localhost:9200")
                        .multiThreaded(true)
			//Per default this implementation will create no more than 2 concurrent connections per given route
			.defaultMaxTotalConnectionPerRoute(<YOUR_DESIRED_LEVEL_OF_CONCURRENCY_PER_ROUTE>)
			// and no more 20 connections in total
			.maxTotalConnection(<YOUR_DESIRED_LEVEL_OF_CONCURRENCY_TOTAL>)
                        .build());
 JestClient client = factory.getObject();
```
> `JestClient` is designed to be singleton, don't construct it for each request!

### Creating an Index
{'fraction_non_alphanumeric': 0.12459353072052028, 'fraction_numerical': 0.008386103029265788, 'mean_word_length': 4.055363321799308, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 95, 'https://': 8, 'lorem ipsum': 0, 'www.': 6, 'xml': 12}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6836134', 'n_tokens_mistral': 1885, 'n_tokens_neox': 1738, 'n_words': 509}