<commit_before><!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <title>Elasticsearch Cheat Sheet for developers</title>

    <meta name="description" content="Elasticsearch API cheatsheet for developers with copy and paste example for the most useful APIs">
    <link rel="shortcut icon" href="./favicon.ico"/>

    <link rel="stylesheet" href="stylesheets/styles.css">
    <link rel="stylesheet" href="stylesheets/pygment_trac.css">
    <script src="javascripts/scale.fix.js"></script>
    <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
    <!--[if lt IE 9]>
    <script src="//html5shiv.googlecode.com/svn/trunk/html5.js"></script>
    <![endif]-->
  </head>
  <body>
    <div class="wrapper">
      <header>
        <h1 class="header">Elasticsearch Cheat&nbsp;Sheet</h1>
        <p class="header">All the API endpoints and pro-tips you always forgot about in one place!</p>

        <div id="menuHolder"></div>

        <ul class="buttonsList">
          <!--
          Todo, maybe a PDF to download?
          <li class="download"><a class="buttons" href="https://github.com/jolicode/elasticsearch-sheatsheet/zipball/master">Download ZIP</a></li>
          <li class="download"><a class="buttons" href="https://github.com/jolicode/elasticsearch-sheatsheet/tarball/master">Download TAR</a></li>-->
          <li><a class="buttons" href="http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index.html">Official Reference</a></li>
          <li><a class="buttons github" href="https://github.com/jolicode/elasticsearch-sheatsheet">View On GitHub</a></li>
        </ul>

        <p class="header">This project is maintained by <a class="header name" href="http://jolicode.com">JoliCode</a>. Found a typo?
        Something missing? We ♥ pull requests!</p>
      </header>

      <section>
        <h3>Tools</h3>

          <p>Forget about CURL and get a <a href="http://www.elasticsearch.org/guide/en/marvel/current/#_sense">real query tool like Sense</a>.</p>

          <p>All queries here and in the <a href="http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/index.html">official guide</a>
            are simplified to a Sense copy and paste line like this:</p>

          <pre><code>curl -XGET 'http://localhost:9200/blog/_search?q=user:foobar'</code></pre>
          <pre><code>GET /blog/_search?q=user:foobar</code></pre>

        <h3>Index API</h3>

        <h4>Create an index (you can also just start indexing documents)</h4>
        <pre><code>PUT /blog
{
  "settings" :
  {
    "number_of_shards" : 3,
    "number_of_replicas" : 1
  },
  "mappings" : ...
}</code></pre>

        <h4>Get the mapping</h4>

        <pre><code>GET /blog/_mapping/post</code></pre>

        <h4>Send the mapping</h4>

        <pre><code>PUT /blog
{
  "mappings": {
    "post" : {
      "properties" : {
        "title" : {
          "type" : "string",
          "analyzer": "english",
          "fields" : {
            "raw" : {
              "type" : "string",
              "index" : "not_analyzed"
            }
          }
        },
        "tag" : {
          "type" : "string",
          "index" : "not_analyzed"
        }
      }
    }
  }
}</code></pre>

        <h4>Open and close index to update settings</h4>
        <pre><code>POST /blog/_close</code></pre>
        <pre><code>POST /blog/_open</code></pre>

        <h4>Test analyzer token output by field</h4>

        <pre><code>GET /blog/_analyze?field=title&text=powerful</code></pre>

        <h4>Test analyzer token output by analyzer</h4>

        <pre><code>GET /_analyze?analyzer=english&text=powerful</code></pre>

        <h4>Index and update a document</h4>

        <pre><code>PUT /blog/post/1
{
  "title": "<NAME> is a very common name",
  "body": "...",
  "user": "foobar",
  "tag": [
    "awesome",
    "smith"
  ]
}</code></pre>

        <h4>Get a document</h4>

        <pre><code>GET /blog/post/1</code></pre>

        <h4>Delete a document</h4>

        <pre><code>DELETE /blog/post/1</code></pre>

        <h4>Add the foo alias on the blog index</h4>
        <pre><code>PUT /blog/_alias/foo</code></pre>

        <h4>List all indexes with alias foo</h4>
        <pre><code>GET /*/_alias/foo</code></pre>

        <h4>List all alias on index blog</h4>
        <pre><code>GET /blog/_alias/*</code></pre>

        <h4>Switch an alias in one command</h4>

        <pre><code>POST /_aliases
{
  "actions": [
    { "remove":
      { "index": "blog", "alias": "foo" }
    },
    { "add":
      { "index": "blog_v2", "alias": "foo" }
    }
  ]
}</code></pre>

        <h3>Debug API</h3>

        <h4>Get a detailed view of what a query do</h4>

        <pre><code>GET /blog/post/_validate/query?explain
{
  "query": {
    "match": {
      "title": "Smith"
    }
  }
}</code></pre>

        <h4>Get explanation about a document match or not</h4>

        <pre><code>GET /blog/post/1/_explain
{
  "query": {
    "match": {
      "title": "Smith"
    }
  }
}</code></pre>

        <h4>Get explanations about documents _score</h4>

        <pre><code>GET /blog/post/_search?explain&pretty
{
  "query": {
    "match": {
      "title": "Smith"
    }
  }
}</code></pre>

        <h3>Search API</h3>

        <h4>Simple query</h4>
        <pre><code>GET /blog/post/_search
{
  "query": {
    "match": {
      "title": "smith"
    }
  }
}</code></pre>

        <h4>Complete search</h4>

        <p>Here is an example of very complex search with aggregation, highlight, filters, pagination, sort, source... All you can do!</p>

        <pre><code>GET /blog/post/_search
{
  "highlight": {
    "pre_tags" : ["&lt;tag1&gt;"],
    "post_tags" : ["&lt;/tag1&gt;"],
    "fields": {
      "body" : {},
      "title": {}
    }
  },
  "size": 20,
  "from": 100,
  "_source": ["title", "id"],
  "sort": {
    { "post_date" : {"order" : "desc"}},
  }
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "title": "smith"
              }
            }
          ],
          "must_not": [
            {
              "match_phrase": {
                "title": "<NAME>"
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must_not": [
            {
              "missing": {
                "field": "title"
              }
            }
          ]
        }
      }
    }
  },
  "aggs": {
    "my_agg": {
      "terms": {
        "field": "user",
        "size": 10
      }
    }
  }
}</code></pre>

        <h3>Plugins</h3>

        <p>On debian the script is here: <code>/usr/share/elasticsearch/bin/plugin</code>.</p>

        <h4>Install various plugins</h4>
        <pre><code>./bin/plugin --install mobz/elasticsearch-head
./bin/plugin --install lmenezes/elasticsearch-kopf/1.2
./bin/plugin --install elasticsearch/marvel/latest</code></pre>

        <h4>Remove a plugin</h4>

        <pre><code>./bin/plugin --remove</code></pre>

        <h4>List installed plugins</h4>
        <pre><code>./bin/plugin --list</code></pre>
      </section>

      <footer>
        <p><small>Hosted on <a href="http://pages.github.com">GitHub Pages</a>, based on Dinky theme</small></p>

        <p>Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.
          This website is not endorsed of affiliated with Elasticsearch.</p>
      </footer>
    </div>
    <!--[if !IE]><script>fixScale(document);</script><![endif]-->

    <script>
      (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
        (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
        m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
      })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

      ga('create', 'UA-29696798-6', 'auto');
      ga('send', 'pageview');

      /**
       * Build the menu
       */
      var titles = document.getElementsByTagName('h3');
      var menu = document.createElement("ul");

      for(var i=0, len=titles.length; i < len; i++){
        titles[i].setAttribute("id", "goto_title_"+i);

        var menuItem = document.createElement("li");
        menuItem.innerHTML = '<a href="#goto_title_'+i+'">'+titles[i].innerHTML+'</a>';
        menu.appendChild(menuItem);
      }

      document.getElementById("menuHolder").appendChild(menu);
    </script>
  </body>
</html>
<commit_msg>Adds option to list all the installed plugins using rest API<commit_after><!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <title>Elasticsearch Cheat Sheet for developers</title>

    <meta name="description" content="Elasticsearch API cheatsheet for developers with copy and paste example for the most useful APIs">
    <link rel="shortcut icon" href="./favicon.ico"/>

    <link rel="stylesheet" href="stylesheets/styles.css">
    <link rel="stylesheet" href="stylesheets/pygment_trac.css">
    <script src="javascripts/scale.fix.js"></script>
    <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
    <!--[if lt IE 9]>
    <script src="//html5shiv.googlecode.com/svn/trunk/html5.js"></script>
    <![endif]-->
  </head>
  <body>
    <div class="wrapper">
      <header>
        <h1 class="header">Elasticsearch Cheat&nbsp;Sheet</h1>
        <p class="header">All the API endpoints and pro-tips you always forgot about in one place!</p>

        <div id="menuHolder"></div>

        <ul class="buttonsList">
          <!--
          Todo, maybe a PDF to download?
          <li class="download"><a class="buttons" href="https://github.com/jolicode/elasticsearch-sheatsheet/zipball/master">Download ZIP</a></li>
          <li class="download"><a class="buttons" href="https://github.com/jolicode/elasticsearch-sheatsheet/tarball/master">Download TAR</a></li>-->
          <li><a class="buttons" href="http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index.html">Official Reference</a></li>
          <li><a class="buttons github" href="https://github.com/jolicode/elasticsearch-sheatsheet">View On GitHub</a></li>
        </ul>

        <p class="header">This project is maintained by <a class="header name" href="http://jolicode.com">JoliCode</a>. Found a typo?
        Something missing? We ♥ pull requests!</p>
      </header>

      <section>
        <h3>Tools</h3>

          <p>Forget about CURL and get a <a href="http://www.elasticsearch.org/guide/en/marvel/current/#_sense">real query tool like Sense</a>.</p>

          <p>All queries here and in the <a href="http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/index.html">official guide</a>
            are simplified to a Sense copy and paste line like this:</p>

          <pre><code>curl -XGET 'http://localhost:9200/blog/_search?q=user:foobar'</code></pre>
          <pre><code>GET /blog/_search?q=user:foobar</code></pre>

        <h3>Index API</h3>

        <h4>Create an index (you can also just start indexing documents)</h4>
        <pre><code>PUT /blog
{
  "settings" :
  {
    "number_of_shards" : 3,
    "number_of_replicas" : 1
  },
  "mappings" : ...
}</code></pre>

        <h4>Get the mapping</h4>

        <pre><code>GET /blog/_mapping/post</code></pre>

        <h4>Send the mapping</h4>

        <pre><code>PUT /blog
{
  "mappings": {
    "post" : {
      "properties" : {
        "title" : {
          "type" : "string",
          "analyzer": "english",
          "fields" : {
            "raw" : {
              "type" : "string",
              "index" : "not_analyzed"
            }
          }
        },
        "tag" : {
          "type" : "string",
          "index" : "not_analyzed"
        }
      }
    }
  }
}</code></pre>

        <h4>Open and close index to update settings</h4>
        <pre><code>POST /blog/_close</code></pre>
        <pre><code>POST /blog/_open</code></pre>

        <h4>Test analyzer token output by field</h4>

        <pre><code>GET /blog/_analyze?field=title&text=powerful</code></pre>

        <h4>Test analyzer token output by analyzer</h4>

        <pre><code>GET /_analyze?analyzer=english&text=powerful</code></pre>

        <h4>Index and update a document</h4>

        <pre><code>PUT /blog/post/1
{
  "title": "<NAME> is a very common name",
  "body": "...",
  "user": "foobar",
  "tag": [
    "awesome",
    "smith"
  ]
}</code></pre>

        <h4>Get a document</h4>

        <pre><code>GET /blog/post/1</code></pre>

        <h4>Delete a document</h4>

        <pre><code>DELETE /blog/post/1</code></pre>

        <h4>Add the foo alias on the blog index</h4>
        <pre><code>PUT /blog/_alias/foo</code></pre>

        <h4>List all indexes with alias foo</h4>
        <pre><code>GET /*/_alias/foo</code></pre>

        <h4>List all alias on index blog</h4>
        <pre><code>GET /blog/_alias/*</code></pre>

        <h4>Switch an alias in one command</h4>

        <pre><code>POST /_aliases
{
  "actions": [
    { "remove":
      { "index": "blog", "alias": "foo" }
    },
    { "add":
      { "index": "blog_v2", "alias": "foo" }
    }
  ]
}</code></pre>

        <h3>Debug API</h3>

        <h4>Get a detailed view of what a query do</h4>

        <pre><code>GET /blog/post/_validate/query?explain
{
  "query": {
    "match": {
      "title": "Smith"
    }
  }
}</code></pre>

        <h4>Get explanation about a document match or not</h4>

        <pre><code>GET /blog/post/1/_explain
{
  "query": {
    "match": {
      "title": "Smith"
    }
  }
}</code></pre>

        <h4>Get explanations about documents _score</h4>

        <pre><code>GET /blog/post/_search?explain&pretty
{
  "query": {
    "match": {
      "title": "Smith"
    }
  }
}</code></pre>

        <h3>Search API</h3>

        <h4>Simple query</h4>
        <pre><code>GET /blog/post/_search
{
  "query": {
    "match": {
      "title": "smith"
    }
  }
}</code></pre>

        <h4>Complete search</h4>

        <p>Here is an example of very complex search with aggregation, highlight, filters, pagination, sort, source... All you can do!</p>

        <pre><code>GET /blog/post/_search
{
  "highlight": {
    "pre_tags" : ["&lt;tag1&gt;"],
    "post_tags" : ["&lt;/tag1&gt;"],
    "fields": {
      "body" : {},
      "title": {}
    }
  },
  "size": 20,
  "from": 100,
  "_source": ["title", "id"],
  "sort": {
    { "post_date" : {"order" : "desc"}},
  }
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "title": "smith"
              }
            }
          ],
          "must_not": [
            {
              "match_phrase": {
                "title": "<NAME>"
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must_not": [
            {
              "missing": {
                "field": "title"
              }
            }
          ]
        }
      }
    }
  },
  "aggs": {
    "my_agg": {
      "terms": {
        "field": "user",
        "size": 10
      }
    }
  }
}</code></pre>

        <h3>Plugins</h3>

        <p>On debian the script is here: <code>/usr/share/elasticsearch/bin/plugin</code>.</p>

        <h4>Install various plugins</h4>
        <pre><code>./bin/plugin --install mobz/elasticsearch-head
./bin/plugin --install lmenezes/elasticsearch-kopf/1.2
./bin/plugin --install elasticsearch/marvel/latest</code></pre>

        <h4>Remove a plugin</h4>

        <pre><code>./bin/plugin --remove</code></pre>

        <h4>List installed plugins</h4>
        <pre><code>./bin/plugin --list</code></pre>
        <pre><code>GET /_nodes?plugin=true</code></pre>
      </section>

      <footer>
        <p><small>Hosted on <a href="http://pages.github.com">GitHub Pages</a>, based on Dinky theme</small></p>

        <p>Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.
          This website is not endorsed of affiliated with Elasticsearch.</p>
      </footer>
    </div>
    <!--[if !IE]><script>fixScale(document);</script><![endif]-->

    <script>
      (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
        (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
        m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
      })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

      ga('create', 'UA-29696798-6', 'auto');
      ga('send', 'pageview');

      /**
       * Build the menu
       */
      var titles = document.getElementsByTagName('h3');
      var menu = document.createElement("ul");

      for(var i=0, len=titles.length; i < len; i++){
        titles[i].setAttribute("id", "goto_title_"+i);

        var menuItem = document.createElement("li");
        menuItem.innerHTML = '<a href="#goto_title_'+i+'">'+titles[i].innerHTML+'</a>';
        menu.appendChild(menuItem);
      }

      document.getElementById("menuHolder").appendChild(menu);
    </script>
  </body>
</html>

{'fraction_non_alphanumeric': 0.18792374109057194, 'fraction_numerical': 0.0111259199165556, 'mean_word_length': 2.24764772299586, 'pattern_counts': {'":': 104, '<': 487, '<?xml version=': 0, '>': 485, 'https://': 6, 'lorem ipsum': 0, 'www.': 8, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6829812', 'n_tokens_mistral': 6024, 'n_tokens_neox': 5805, 'n_words': 1120}