<commit_before>---
layout: layout.pug
navigationTitle:
excerpt:
title: Install and Customize
menuWeight: 20
---
{% assign data = site.data.services.hdfs %}

{% capture customInstallRequirements %}
- Each agent node must have eight GiB of memory and ten GiB of disk, and each must have these ports available: 8480, 8485, 9000, 9001, 9002, 9005, and 9006, and 9007.
{% endcapture %}

{% capture customInstallConfigurations %}
### Example custom Installation

If you are ready to ship into production, you will likely need to customize the deployment to suit the workload requirements of your application(s). Customize the default deployment by creating a JSON file, then pass it to `dcos package install` using the `--options` parameter.

Sample JSON options file named `sample-{{ data.serviceName }}.json`:

```json
{
  "data_node": {
    "count": 10
  }
}
```

This cluster will have 10 data nodes instead of the default value of 3.
See the Configuration section for a list of fields that can be customized via a options JSON file when the HDFS cluster is created.

### Minimal installation

Many of the other Infinity services currently support deployment in DC/OS Vagrant or DC/OS Docker for local testing a local machine. However, DC/OS HDFS currently only supports deployment with an HA name service managed by a Quorum Journal. The resource requirements for such a deployment make it prohibitive to install on a local development machine. The default deployment, is the minimal safe deployment for a DC/OS HDFS cluster. Community contributions to support deployment of a non-HA cluster, e.g. a single name node and data node with no failure detector, would be welcome.
{% endcapture %}

{% include services/install.md
    data=data
    customInstallRequirements=customInstallRequirements
    customInstallConfigurations=customInstallConfigurations %}

## Topology

An individual HDFS deployment will colocate name nodes with journal nodes, but it will not colocate two name nodes or two journal nodes on the same agent node in the cluster. Data nodes may be colocated with both name nodes and/or journal nodes. If multiple HDFS clusters are installed, they may share the same agent nodes in the cluster provided that no ports specified in the service configurations conflict for those node types.

## Initial Deployment Strategy

HDFS deployment occurs in a series of Phases which are configured in the HDFS service's Deploy Plan.

The `deploy` plan may be viewed either using the DC/OS CLI or directly over HTTP. See the [API Reference](api-reference.md#view-plan) for more information on viewing the progress of the `deploy` plan.
<commit_msg>Update HDFS docs to reflect correct minimum node count and DC/OS Vagrant mentions (#2263)<commit_after>---
layout: layout.pug
navigationTitle:
excerpt:
title: Install and Customize
menuWeight: 20
---
{% assign data = site.data.services.hdfs %}

{% capture customInstallRequirements %}
- Each agent node must have eight GiB of memory and ten GiB of disk, and each must have these ports available: 8480, 8485, 9000, 9001, 9002, 9005, and 9006, and 9007.
{% endcapture %}

{% capture customInstallConfigurations %}
### Example custom Installation

If you are ready to ship into production, you will likely need to customize the deployment to suit the workload requirements of your application(s). Customize the default deployment by creating a JSON file, then pass it to `dcos package install` using the `--options` parameter.

Sample JSON options file named `sample-{{ data.serviceName }}.json`:

```json
{
  "data_node": {
    "count": 10
  }
}
```

This cluster will have 10 data nodes instead of the default value of 3.
See the Configuration section for a list of fields that can be customized via a options JSON file when the HDFS cluster is created.

### Minimal installation

Many of the other Infinity services currently support deployment in DC/OS Docker for local testing a local machine. However, DC/OS HDFS currently only supports deployment with an HA name service managed by a Quorum Journal. The resource requirements for such a deployment make it prohibitive to install on a local development machine. The default deployment, is the minimal safe deployment for a DC/OS HDFS cluster. Community contributions to support deployment of a non-HA cluster, e.g. a single name node and data node with no failure detector, would be welcome.
{% endcapture %}

{% include services/install.md
    data=data
    customInstallRequirements=customInstallRequirements
    customInstallConfigurations=customInstallConfigurations %}

## Topology

An individual HDFS deployment will colocate name nodes with journal nodes, but it will not colocate two name nodes or two journal nodes on the same agent node in the cluster. Data nodes may be colocated with both name nodes and/or journal nodes. If multiple HDFS clusters are installed, they may share the same agent nodes in the cluster provided that no ports specified in the service configurations conflict for those node types.

## Initial Deployment Strategy

HDFS deployment occurs in a series of Phases which are configured in the HDFS service's Deploy Plan.

The `deploy` plan may be viewed either using the DC/OS CLI or directly over HTTP. See the [API Reference](api-reference.md#view-plan) for more information on viewing the progress of the `deploy` plan.

{'fraction_non_alphanumeric': 0.055793991416309016, 'fraction_numerical': 0.01530136219443926, 'mean_word_length': 4.577523413111343, 'pattern_counts': {'":': 4, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4530401', 'n_tokens_mistral': 1411, 'n_tokens_neox': 1275, 'n_words': 752}