<commit_before>[[getting-started]]
== Getting Started with Beats and the Elastic Stack 

Looking for an "ELK tutorial" that shows how to set up the Elastic stack for Beats? You've
come to the right place. The topics in this section describe how to install and configure
the Elastic stack for Beats.

A regular _Beats setup_ consists of:

 * One or more Beats. You install the Beats on your servers to capture operational data.
 * Elasticsearch for storage and indexing. See <<elasticsearch-installation>>.
 * Logstash (optional) for inserting data into Elasticsearch. See <<logstash-installation,Installing Logstash>>.
 * Kibana for the UI. See <<kibana-installation>>.
 * Kibana dashboards for visualizing the data. See <<load-kibana-dashboards>>.
 
See the https://www.elastic.co/support/matrix[Elastic Support Matrix] for information 
about supported operating systems and product compatibility.

NOTE: To get started, you can install Elasticsearch and Kibana on a
single VM or even on your laptop. The only condition is that the machine must be
accessible from the servers you want to monitor. As you add more Beats and
your traffic grows, you'll want to replace the single Elasticsearch instance with
a cluster. You'll probably also want to automate the installation process.


[[elasticsearch-installation]]
=== Installing Elasticsearch

https://www.elastic.co/products/elasticsearch[Elasticsearch] is a real-time,
distributed storage, search, and analytics engine. It can be used for many
purposes, but one context where it excels is indexing streams of semi-structured
data, such as logs or decoded network packets.

The binary packages of Elasticsearch have only one dependency: Java. The minimum
supported version is Java 7. To download and install
Elasticsearch, use the commands that work with your system
(<<deb, deb>> for Debian/Ubuntu, <<rpm, rpm>> for Redhat/Centos/Fedora, <<mac,
mac>> for OS X, and <<win, win>> for Windows):

[[deb]]*deb:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo apt-get install openjdk-7-jre
curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/{ES-version}/elasticsearch-{ES-version}.deb
sudo dpkg -i elasticsearch-{ES-version}.deb
sudo /etc/init.d/elasticsearch start
----------------------------------------------------------------------

[[rpm]]*rpm:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo yum install java-1.7.0-openjdk
curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/{ES-version}/elasticsearch-{ES-version}.rpm
sudo rpm -i elasticsearch-{ES-version}.rpm
sudo service elasticsearch start
----------------------------------------------------------------------

[[mac]]*mac:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
# install Java, e.g. from: https://www.java.com/en/download/manual.jsp
curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/{ES-version}/elasticsearch-{ES-version}.zip
unzip elasticsearch-{ES-version}.zip
cd elasticsearch-{ES-version}
./bin/elasticsearch
----------------------------------------------------------------------

[[win]]*win:*

. If necessary, download and install the latest version of the Java from https://www.java.com[www.java.com].

. Download the Elasticsearch {ES-version} Windows zip file from the
https://www.elastic.co/downloads/elasticsearch[downloads page].

. Extract the contents of the zip file to a directory on your computer, for example, `C:\Program Files`.

. Open a command prompt as an Administrator and navigate to the directory that contains the extracted files, for example:
+
["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
cd C:\Program Files\elasticsearch-{ES-version}
----------------------------------------------------------------------

. Run the following command to start Elasticsearch:
+
["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
bin\elasticsearch.bat
----------------------------------------------------------------------

You can learn more about installing, configuring, and running Elasticsearch in the
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html[Elasticsearch Reference].

==== Making Sure Elasticsearch is Up and Running


To test that the Elasticsearch daemon is up and running, try sending an HTTP GET
request on port 9200.

[source,shell]
----------------------------------------------------------------------
curl http://127.0.0.1:9200
----------------------------------------------------------------------

On Windows, if you don't have cURL installed, simply point your browser to the URL.

You should see a response similar to this:

[source,shell]
----------------------------------------------------------------------
{
  "name" : "<NAME>",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.2.0",
    "build_hash" : "8ff36d139e16f8720f2947ef62c8167a888992fe",
    "build_timestamp" : "2016-01-27T13:32:39Z",
    "build_snapshot" : false,
    "lucene_version" : "5.4.1"
  },
  "tagline" : "You Know, for Search"
}
----------------------------------------------------------------------

[[logstash-installation]]
=== Installing Logstash (Optional)

The simplest architecture for the Beats platform setup consists of one or more Beats,
Elasticsearch, and Kibana. This architecture is easy to get started
with and sufficient for networks with low traffic. It also uses the minimum amount of
servers: a single machine running Elasticsearch and Kibana. The Beats
insert the transactions directly into the Elasticsearch instance. 

If you want to perform additional processing or buffering on the data, however,
you'll want to install Logstash. 

An important advantage to this approach is that you can
use Logstash to modify the data captured by Beats in any way you like. You can also
use Logstash's many output plugins to integrate with other systems.

image:./images/beats-logstash.png[Integration with Logstash]

To download and install Logstash, use the commands that work
with your system:

*deb:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo apt-get install openjdk-7-jre
curl -L -O https://download.elastic.co/logstash/logstash/packages/debian/logstash-{LS-version}_all.deb
sudo dpkg -i logstash-{LS-version}_all.deb
----------------------------------------------------------------------

*rpm:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo yum install java-1.7.0-openjdk
curl -L -O https://download.elastic.co/logstash/logstash/packages/centos/logstash-{LS-version}.noarch.rpm
sudo rpm -i logstash-{LS-version}.noarch.rpm
----------------------------------------------------------------------

*mac:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
# install Java, e.g. from: https://www.java.com/en/download/manual.jsp
curl -L -O https://download.elastic.co/logstash/logstash/logstash-{LS-version}.zip<commit_msg>Update Java version to 8 (#1693)<commit_after>[[getting-started]]
== Getting Started with Beats and the Elastic Stack 

Looking for an "ELK tutorial" that shows how to set up the Elastic stack for Beats? You've
come to the right place. The topics in this section describe how to install and configure
the Elastic stack for Beats.

A regular _Beats setup_ consists of:

 * One or more Beats. You install the Beats on your servers to capture operational data.
 * Elasticsearch for storage and indexing. See <<elasticsearch-installation>>.
 * Logstash (optional) for inserting data into Elasticsearch. See <<logstash-installation,Installing Logstash>>.
 * Kibana for the UI. See <<kibana-installation>>.
 * Kibana dashboards for visualizing the data. See <<load-kibana-dashboards>>.
 
See the https://www.elastic.co/support/matrix[Elastic Support Matrix] for information 
about supported operating systems and product compatibility.

NOTE: To get started, you can install Elasticsearch and Kibana on a
single VM or even on your laptop. The only condition is that the machine must be
accessible from the servers you want to monitor. As you add more Beats and
your traffic grows, you'll want to replace the single Elasticsearch instance with
a cluster. You'll probably also want to automate the installation process.


[[elasticsearch-installation]]
=== Installing Elasticsearch

https://www.elastic.co/products/elasticsearch[Elasticsearch] is a real-time,
distributed storage, search, and analytics engine. It can be used for many
purposes, but one context where it excels is indexing streams of semi-structured
data, such as logs or decoded network packets.

The binary packages of Elasticsearch have only one dependency: Java. The minimum
supported version is Java 8. To download and install
Elasticsearch, use the commands that work with your system
(<<deb, deb>> for Debian/Ubuntu, <<rpm, rpm>> for Redhat/Centos/Fedora, <<mac,
mac>> for OS X, and <<win, win>> for Windows):

[[deb]]*deb:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo apt-get install openjdk-8-jre
curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/{ES-version}/elasticsearch-{ES-version}.deb
sudo dpkg -i elasticsearch-{ES-version}.deb
sudo /etc/init.d/elasticsearch start
----------------------------------------------------------------------

[[rpm]]*rpm:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo yum install java-1.8.0-openjdk
curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/{ES-version}/elasticsearch-{ES-version}.rpm
sudo rpm -i elasticsearch-{ES-version}.rpm
sudo service elasticsearch start
----------------------------------------------------------------------

[[mac]]*mac:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
# install Java, e.g. from: https://www.java.com/en/download/manual.jsp
curl -L -O https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/{ES-version}/elasticsearch-{ES-version}.zip
unzip elasticsearch-{ES-version}.zip
cd elasticsearch-{ES-version}
./bin/elasticsearch
----------------------------------------------------------------------

[[win]]*win:*

. If necessary, download and install the latest version of the Java from https://www.java.com[www.java.com].

. Download the Elasticsearch {ES-version} Windows zip file from the
https://www.elastic.co/downloads/elasticsearch[downloads page].

. Extract the contents of the zip file to a directory on your computer, for example, `C:\Program Files`.

. Open a command prompt as an Administrator and navigate to the directory that contains the extracted files, for example:
+
["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
cd C:\Program Files\elasticsearch-{ES-version}
----------------------------------------------------------------------

. Run the following command to start Elasticsearch:
+
["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
bin\elasticsearch.bat
----------------------------------------------------------------------

You can learn more about installing, configuring, and running Elasticsearch in the
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html[Elasticsearch Reference].

==== Making Sure Elasticsearch is Up and Running


To test that the Elasticsearch daemon is up and running, try sending an HTTP GET
request on port 9200.

[source,shell]
----------------------------------------------------------------------
curl http://127.0.0.1:9200
----------------------------------------------------------------------

On Windows, if you don't have cURL installed, simply point your browser to the URL.

You should see a response similar to this:

[source,shell]
----------------------------------------------------------------------
{
  "name" : "<NAME>",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.2.0",
    "build_hash" : "8ff36d139e16f8720f2947ef62c8167a888992fe",
    "build_timestamp" : "2016-01-27T13:32:39Z",
    "build_snapshot" : false,
    "lucene_version" : "5.4.1"
  },
  "tagline" : "You Know, for Search"
}
----------------------------------------------------------------------

[[logstash-installation]]
=== Installing Logstash (Optional)

The simplest architecture for the Beats platform setup consists of one or more Beats,
Elasticsearch, and Kibana. This architecture is easy to get started
with and sufficient for networks with low traffic. It also uses the minimum amount of
servers: a single machine running Elasticsearch and Kibana. The Beats
insert the transactions directly into the Elasticsearch instance. 

If you want to perform additional processing or buffering on the data, however,
you'll want to install Logstash. 

An important advantage to this approach is that you can
use Logstash to modify the data captured by Beats in any way you like. You can also
use Logstash's many output plugins to integrate with other systems.

image:./images/beats-logstash.png[Integration with Logstash]

To download and install Logstash, use the commands that work
with your system:

*deb:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo apt-get install openjdk-8-jre
curl -L -O https://download.elastic.co/logstash/logstash/packages/debian/logstash-{LS-version}_all.deb
sudo dpkg -i logstash-{LS-version}_all.deb
----------------------------------------------------------------------

*rpm:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
sudo yum install java-1.8.0-openjdk
curl -L -O https://download.elastic.co/logstash/logstash/packages/centos/logstash-{LS-version}.noarch.rpm
sudo rpm -i logstash-{LS-version}.noarch.rpm
----------------------------------------------------------------------

*mac:*

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
# install Java, e.g. from: https://www.java.com/en/download/manual.jsp
curl -L -O https://download.elastic.co/logstash/logstash/logstash-{LS-version}.zip
{'fraction_non_alphanumeric': 0.2660476922049295, 'fraction_numerical': 0.0098189833678445, 'mean_word_length': 2.141418380193034, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 26, 'lorem ipsum': 0, 'www.': 16, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 28, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '913296', 'n_tokens_mistral': 3849, 'n_tokens_neox': 3671, 'n_words': 1300}