<commit_before>---
layout: post
title: "Data visualization using Elasticsearch Logstash and Kibana"
description: "Measure execution time of small code"
comments: true
keywords: "python, execution time, alogorithms, Ipython"
---

Elasticsearch Logstash and Kibana popularly abbreviated as ELK is a modern data aggregation, data querying and data visualization tool.
All the compnents of the ELK stack are seamlessly integrated for working coherently. ELK stack is fast gaining popularity and is being adopted by
corporate conglomerates too. It has a simple setting up process and even simpler learning curve. I will introduce in brief the three components of ELK stack.

To make the process of using ELK stack even clearer I would be visualizing the [Sacramento Real estate transaction data](https://support.spatialkey.com/spatialkey-sample-csv-data/)


### Elasticsearch

Elasticsearch is a schema free search engine, based on Lucene. It provides various REST APIs to query across data. It is fast because it queries
data based on Index. It is like searching the contents of book through its index. Data in index is organized on basis of documents.An index can contain
multiple documents. Elasticsearch centrally stores our data.
An example of its REST API is the *GET* API that searches across multiple or unique indexes.
It can executed through the console/command line via *CURL* or one can use [POSTMAN](https://chrome.google.com/webstore/detail/postman/fhbjgbiflinjbdggehcddcbncdddomop?hl=en).

CURL syntax is as follows for a simple *GET* query:
    
    `curl -XGET 'localhost:9200/twitter/tweet/0?pretty'`

It returns a json document for the index `twitter` and document id `tweet`




### Logstash


### Kibana






<commit_msg>order changed of the log<commit_after>---
layout: post
title: "Data visualization using Elasticsearch Logstash and Kibana"
description: "Measure execution time of small code"
comments: true
keywords: "python, execution time, alogorithms, Ipython"
---

Elasticsearch Logstash and Kibana popularly abbreviated as ELK is a modern data aggregation, data querying and data visualization tool.
All the compnents of the ELK stack are seamlessly integrated for working coherently. ELK stack is fast gaining popularity and is being adopted by
corporate conglomerates too. It has a simple setting up process and even simpler learning curve. I will introduce in brief the three components of ELK stack.

To make the process of using ELK stack even clearer I would be visualizing the [Sacramento Real estate transaction data](https://support.spatialkey.com/spatialkey-sample-csv-data/)


### Logstash


### Elasticsearch

Elasticsearch is a schema free search engine, based on Lucene. It provides various REST APIs to query across data. It is fast because it queries
data based on Index. It is like searching the contents of book through its index. Data in index is organized on basis of documents.An index can contain
multiple documents. Elasticsearch centrally stores our data.
An example of its REST API is the *GET* API that searches across multiple or unique indexes.
It can executed through the console/command line via *CURL* or one can use [POSTMAN](https://chrome.google.com/webstore/detail/postman/fhbjgbiflinjbdggehcddcbncdddomop?hl=en).

CURL syntax is as follows for a simple *GET* query:
    
    `curl -XGET 'localhost:9200/twitter/tweet/0?pretty'`

It returns a json document for the index `twitter` and document id `tweet`



### Kibana







{'fraction_non_alphanumeric': 0.05679862306368331, 'fraction_numerical': 0.002868617326448652, 'mean_word_length': 4.624193548387097, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1557741', 'n_tokens_mistral': 970, 'n_tokens_neox': 908, 'n_words': 472}