<commit_before># Import data from MSSQL into Elasticsearch  #

**Prerequisite:**

1. Ubuntu 14.04
2. Install [Docker Host](https://docs.docker.com/installation/ubuntulinux/)
3. Install [elasticsearch](https://registry.hub.docker.com/_/elasticsearch/) 
	
		docker run -d -p 9200:9200 -p 9300:9300 elasticsearch 

4. Install [Kibana](http://git.tavisca.com/vbhadage/docker-kibana)
	1. check out 
	
			git clone http://git.tavisca.com/vbhadage/docker-kibana.git

	2. Build the image

			git build -t doker-kibana .

	3. Run the kibana with Elastic search image

			docker run -d -e ELASTICSEARCH_PORT_9200_TCP_ADDR=***your-ES-IP-or-host-name** -p 5601:5601 docker-kibana
	




**Step 1:** Check out the docker file from **git.tavisca.com**

	git clone http://git.tavisca.com/vbhadage/data-importer.git


**Step 2:** Build the images from Dockerfile

	docker build -t data-importer .

**Step 3:** Run the data-importer by setting following parameter 


 1. **LAST_EXECUTION_START**="2014-06-06T09:08:00.948Z"
    
	This data time used to import the data from MS-SQL audit log table. 
 All records in auditlog table with timestamp greater than this will be imported in Elastic search.

 2. **INDEX_NAME**=** Provide the value **

This one is index name for elasticsearch.

 3. **CLUSTER**=** Provide the value **
 
Provide the elasticsearch cluster name.

 4. **ES_HOST**=** Provide the value **
 <commit_msg>Changed the path<commit_after># Import data from MS sql Server into Elasticsearch 1.6#

**Problem:**

Need to provide the analytics and visualization for audit log data which is stored in relational database. 
 
**Solution**

one of the solution to this problem is to visualize the data in open source tool like [kibana](https://www.elastic.co/products/kibana). But kibana uses the [elasticsearch](https://www.elastic.co/products/elasticsearch) for search and storage purpose.

So that need to import selected records from relational database into the elasticsearch 1.6. The new index will be created in elasticsearch for this data and it will be used kibana.

Prior to elasticsearch 1.6 the [river plugin](https://www.elastic.co/blog/deprecating-rivers) was available for this purpose but it is now deprecated.

But to solve the same problem another standalone java utility known as [elasticsearch - jdbc](https://github.com/jprante/elasticsearch-jdbc) is available. 

Here I am going to tell you how to use this utility through [docker](https://www.docker.com/) so whenever you need it. it would be only three steps process for you i.e clone it, build image and start the container with parameter. 



**Prerequisite:**

1. Ubuntu 14.04
2. Install [Docker Host](https://docs.docker.com/installation/ubuntulinux/)
3. Install [elasticsearch](https://registry.hub.docker.com/_/elasticsearch/) 
	
		docker run -d -p 9200:9200 -p 9300:9300 elasticsearch 

4. Install [Kibana](https://www.elastic.co/products/kibana)
	1. check out 
	
			git clone https://github.com/vinayakbhadage/docker-kibana.git

	2. Build the image

			git build -t doker-kibana .

	3. Run the kibana with Elastic search image

			docker run -d -e ELASTICSEARCH_PORT_9200_TCP_ADDR=***your-ES-IP-or-host-name** -p 5601:5601 docker-kibana
	




**Step 1:** Check out the docker file [https://github.com/vinayakbhadage/data-importer](https://github.com/vinayakbhadage/data-importer)

	git clone https://github.com/vinayakbhadage/data-importer.git


**Step 2:** Build the images from Dockerfile

	docker build -t data-importer .

**Step 3:** Run the data-importer by setting following parameter 


 1. **LAST_EXECUTION_START**="2014-06-06T09:08:00.948Z"
    
	This data time used to import the data from MS-SQL audit log table. 
 All records in auditlog table with timestamp greater than this will be imported in Elastic search.

 2. **INDEX_NAME**=** Provide the value **

This one is index name for elasticsearch.

 3. **CLUSTER**=** Provide the value **
 
Provide the elasticsearch cluster name.

 4. **ES_HOST**=** Provide the value **
 
{'fraction_non_alphanumeric': 0.10678642714570859, 'fraction_numerical': 0.03293413173652695, 'mean_word_length': 3.9190184049079755, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 14, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1581836', 'n_tokens_mistral': 1421, 'n_tokens_neox': 1241, 'n_words': 452}