<commit_before># `elasticsearch-tableau-connector`
Tableau Web Data Connector for Elasticsearch

# Overview

This is an instance of a Tableau Web Data Connector for Elasticsearch.  This will extract a set of data from Elasticsearch
based on the cluster URL, index name, type and optional custom query and create a Tableau extract of the data.  The connector
will query the specified type's mapping in Elasticsearch to report the fields and data types that Tableau
can expect to see.

This connector should be periodically refreshed, as the extract only includes data from the point in time 
that it was executed.

The connector works by retrieving 'pages' of data from Elasticsearch up to either the limit specified, or up to the total
number of hits.  The user can override the batch size to retrieve more records per page if desired.

# Known Issues and Limitations

- At this time there appears that the Tableau SDK test framework that executes the connector will only make 6 requests for pages
of data in Elasticsearch, after this a `Maximum Number of Requests Reached` error will be logged
- Fields with `array` or`object` they will be ignored

# Configuration
- You must enable CORS support in your Elasticsearch server.  Set the following setting in `elasticsearch.yml`:

```yaml
http.cors.enabled = true
```

[Elasticsearch Configuration Reference](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html)

If you use some other front end HTTP proxy in front of your Elasticsearch cluster, you will need to make sure that
CORS requests are allowed, including authorization headers.

# Building and Running

## Creating distribution suitable for deploying to Tableau Server
From the command line execute:
```
grunt build:dist
```

This will package the connector files in the `dist` folder, combining javascript and CSS into single files.

You can run the application at the command line with grunt with:
```
grunt
```

This will watch all sub-directories for changes and reload the application if anything changes.  Running this app will
simpy host all the connector resources but when requested stand alone will not do anything useful.  You should either 
use the connector within the [Web Data Connector SDK](http://community.tableau.com/thread/178867)
test harness, or use the connector from Tableau Desktop or Server.

> Note that internally there are tasks that run the `build:dev` target, to perform HTML templating, and copy all
source files to the `public/` source folder where the NodeJS server will serve static resources from

# Usage

## Using in Tableau Desktop

Build the application first:
```
grunt build:dev
```

Run the application, from the command line execute:

```
node index.js
```

or from a development machine, you can run:
```
grunt
```

> This will serve the static resources from the `public/` sub folder.

Make note of the URL that the connector app is running on, e.g.:
```
Elasticsearch Tableau Web Data connector server listening at http://0.0.0.0:3000
```

Simply choose the 'Web Data Connector' as your data source and enter the URL (you might need to enter
the IP address or host name).

## Importing into Tableau Server

Execute the build for this project from the command line:

```
grunt build:dist
```

For each file in the `dist/` folder, import into Tableau Server by:

- Ensure the Tableau command line tools are in your PATH
- From a command line (and your working director is the `dist/` folder) execute the following:

```
tabadmin import_webdataconnector elasticsearch-connector.html
tabadmin import_webdataconnector elasticsearch-connector.min.css
tabadmin import_webdataconnector elasticsearch-connector.js
```

Get the URL of the `elasticsearch-connector.html` on the Tableau Server by executing:
```
tabadmin list_webdataconnectors --urls
```

And from Tableau go to 'Web Data Connector' and enter the URL of the connector:

![](https://github.com/mradamlacey/elasticsearch-tableau-connector/blob/master/resources/wdc_desktop_use_connector.png)

```
http://<your tableau server>/webdataconnectors/elasticsearch-connector.html
```

## Using with the Web Data Connector SDK
If you are running this web app locally, and testing from the Tableau Web Data Connector SDK, simply enter:<commit_msg>Correct readme on instructions for installing to Tableau server<commit_after># `elasticsearch-tableau-connector`
Tableau Web Data Connector for Elasticsearch

# Overview

This is an instance of a Tableau Web Data Connector for Elasticsearch.  This will extract a set of data from Elasticsearch
based on the cluster URL, index name, type and optional custom query and create a Tableau extract of the data.  The connector
will query the specified type's mapping in Elasticsearch to report the fields and data types that Tableau
can expect to see.

This connector should be periodically refreshed, as the extract only includes data from the point in time 
that it was executed.

The connector works by retrieving 'pages' of data from Elasticsearch up to either the limit specified, or up to the total
number of hits.  The user can override the batch size to retrieve more records per page if desired.

# Known Issues and Limitations

- At this time there appears that the Tableau SDK test framework that executes the connector will only make 6 requests for pages
of data in Elasticsearch, after this a `Maximum Number of Requests Reached` error will be logged
- Fields with `array` or`object` they will be ignored

# Configuration
- You must enable CORS support in your Elasticsearch server.  Set the following setting in `elasticsearch.yml`:

```yaml
http.cors.enabled = true
```

[Elasticsearch Configuration Reference](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html)

If you use some other front end HTTP proxy in front of your Elasticsearch cluster, you will need to make sure that
CORS requests are allowed, including authorization headers.

# Building and Running

## Creating distribution suitable for deploying to Tableau Server
From the command line execute:
```
grunt build:dist
```

This will package the connector files in the `dist` folder, combining javascript and CSS into single files.

You can run the application at the command line with grunt with:
```
grunt
```

This will watch all sub-directories for changes and reload the application if anything changes.  Running this app will
simpy host all the connector resources but when requested stand alone will not do anything useful.  You should either 
use the connector within the [Web Data Connector SDK](http://community.tableau.com/thread/178867)
test harness, or use the connector from Tableau Desktop or Server.

> Note that internally there are tasks that run the `build:dev` target, to perform HTML templating, and copy all
source files to the `public/` source folder where the NodeJS server will serve static resources from

# Usage

## Using in Tableau Desktop

Build the application first:
```
grunt build:dev
```

Run the application, from the command line execute:

```
node index.js
```

or from a development machine, you can run:
```
grunt
```

> This will serve the static resources from the `public/` sub folder.

Make note of the URL that the connector app is running on, e.g.:
```
Elasticsearch Tableau Web Data connector server listening at http://0.0.0.0:3000
```

Simply choose the 'Web Data Connector' as your data source and enter the URL (you might need to enter
the IP address or host name).

## Importing into Tableau Server

Execute the build for this project from the command line:

```
grunt build:dist
```

For each file in the `dist/` folder, import into Tableau Server by:

- Ensure the Tableau command line tools are in your PATH
- From a command line (and your working director is the `dist/` folder) execute the following:

```
tabadmin import_webdataconnector elasticsearch-connector.html
tabadmin import_webdataconnector elasticsearch-connector.min.css
tabadmin import_webdataconnector elasticsearch-connector.min.js
```

Get the URL of the `elasticsearch-connector.html` on the Tableau Server by executing:
```
tabadmin list_webdataconnectors --urls
```

And from Tableau go to 'Web Data Connector' and enter the URL of the connector:

![](https://github.com/mradamlacey/elasticsearch-tableau-connector/blob/master/resources/wdc_desktop_use_connector.png)

```
http://<your tableau server>/webdataconnectors/elasticsearch-connector.html
```

## Using with the Web Data Connector SDK
If you are running this web app locally, and testing from the Tableau Web Data Connector SDK, simply enter:
{'fraction_non_alphanumeric': 0.06053915872646991, 'fraction_numerical': 0.0034859400418312805, 'mean_word_length': 4.7151394422310755, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 9, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3644368', 'n_tokens_mistral': 2307, 'n_tokens_neox': 2190, 'n_words': 1173}