<commit_before>## Apache Hue

[![Build Status](https://travis-ci.org/MetaCenterCloudPuppet/cesnet-hue.svg?branch=master)](https://travis-ci.org/MetaCenterCloudPuppet/cesnet-hue)

#### Table of Contents

1. [Module Description - What the module does and why it is useful](#module-description)
2. [Setup - The basics of getting started with hue](#setup)
    * [What hue affects](#what-hue-affects)
    * [Setup requirements](#setup-requirements)
3. [Usage - Configuration options and additional functionality](#usage)
    * [Basic cluster usage](#basic-cluster-usage)
    * [High availability cluster usage](#high-availability-cluster-usage)
    * [Enable security](#enable-security)
    * [Enable SPNEGO authentization](#enable-spnego-authentization)
4. [Reference - An under-the-hood peek at what the module is doing and how](#reference)
    * [Classes](#classes)
    * [Module Parameters (hue class)](#class-hue)
5. [Limitations - OS compatibility, etc.](#limitations)
6. [Development - Guide for contributing to the module](#development)

## Module Description

Installs Apache Hue - web user interface for Hadoop environment.

## Setup

### What hue affects

* Alternatives:
 * alternatives are used for */etc/hue/conf* in Cloudera
 * this module switches to the new alternative by default, so the original configuration can be kept intact
* Files modified:
 * */etc/hue/conf/hue.ini*
 * */etc/security/keytab/hue.service.keytab* ownership changed (only when security is enabled, the path can be changed by *keytab_hue* parameter)
 * */etc/grid-security/hostcert.pem* copied to */etc/hue/conf/* (only with *https*)
 * */etc/grid-security/hostkey.pem* copied to */etc/hue/conf/* (only with *https*)
* Packages: *hue*
* Services: *hue*
* Users and groups:
 * *hue::hdfs* and *hue::user* classes creates the user *hue* and group *hue*

### Setup Requirements

* Hadoop cluster with WebHDFS or httpfs (httpfs is required for HDFS HA)
* HBase with thrift server (optional)
* Hive Server2 (optional)
* Authorizations set ('hue' group must be enabled in *security.client.protocol.acl*, default '*' is OK)
* Proxy users set for Hadoop core, Hadoop httpfs, Ozzie server
 * for *cesnet::hadoop* puppet module: parameters *hue\_hostnames* and *httpfs\_hostnames*
 * for *cesnet::oozie* puppet module: parameter *hue\_hostnames*

## Usage

### Basic cluster usage

    $master_hostname = 'hdfs.example.com'
    $hue_hostname = 'hue.example.com'

    class { '::hadoop':
      ...
      hue_hostnames    => ['hue.example.com'],
    }

    node 'hdfs.example.com' {
      include ::hadoop::namenode
      ...
      include ::hue::hdfs
    }

    node 'hue.example.com' {
      class { '::hue':
        hdfs_hostname => $master_hostname,
        #yarn_hostname  => ...,
        #oozie_hostname => ...,
      }
    }

### High availability cluster usage

    $cluster_name = 'cluster',
    $master_hostnames = [
      'master1.example.com',
      'master2.example.com',
    ]
    $hue_hostname = 'hue.example.com'

    class { '::hadoop':
      ...
      cluster_name      => $cluster_name,
      hdfs_hostname    => $master_hostnames[0],
      hdfs_hostnames   => $master_hostnames[1],
      hue_hostnames    => ['hue.example.com'],
    }

    node 'master1.example.com' {
      include ::hadoop::namenode
      ...
      include ::hue::hdfs
    }

    node 'master2.example.com' {
      include ::hadoop::namenode
      ...
      include ::hue::user
    }

    node 'hue.example.com' {
      class { '::hue':
        defaultFS       => "hdfs://${cluster_name}",
        httpfs_hostname => $hue_hostname,
        #yarn_hostname  => ...,
        #yarn_hostname2 => ...,
        #oozie_hostname => ...,
      }
    }

There is also needed class *hue::hdfs* on all HDFS Namenodes to authorization work properly. You can use *hue::user* instead, or install *hue-common* package.

It is recommended to set properties *hadoop.yarn_clusters.default.logical_name* and *hadoop.yarn_clusters.ha.logical_name* according to the *yarn.resourcemanager.ha.rm-ids* from Hadoop YARN. *cesnet-hue* module uses 'rm1' and 'rm2' values, which is *cesnet-hadoop* puppet module default.

### Enable security

Useful parameters: [*https*](#https) [*https\_cachain*](#https_cachain) [*https\_certificate*](#https_certificate) [*https\_private\_key*](#https_private_key) [*https\_passphrase*](#https_passphrase) [*keytab\_hue*](#keytab_hue) [*realm*](#realm)<commit_msg>Documentation: small improvements, add some secrets.<commit_after>## Apache Hue Web Interface

[![Build Status](https://travis-ci.org/MetaCenterCloudPuppet/cesnet-hue.svg?branch=master)](https://travis-ci.org/MetaCenterCloudPuppet/cesnet-hue)

#### Table of Contents

1. [Module Description - What the module does and why it is useful](#module-description)
2. [Setup - The basics of getting started with hue](#setup)
    * [What hue affects](#what-hue-affects)
    * [Setup requirements](#setup-requirements)
3. [Usage - Configuration options and additional functionality](#usage)
    * [Basic cluster usage](#basic-cluster-usage)
    * [High availability cluster usage](#high-availability-cluster-usage)
    * [Enable security](#enable-security)
    * [Enable SPNEGO authentization](#enable-spnego-authentization)
4. [Reference - An under-the-hood peek at what the module is doing and how](#reference)
    * [Classes](#classes)
    * [Module Parameters (hue class)](#class-hue)
5. [Limitations - OS compatibility, etc.](#limitations)
6. [Development - Guide for contributing to the module](#development)

## Module Description

Installs Apache Hue - web user interface for Hadoop environment.

## Setup

### What hue affects

* Alternatives:
 * alternatives are used for */etc/hue/conf* in Cloudera
 * this module switches to the new alternative by default, so the original configuration can be kept intact
* Files modified:
 * */etc/hue/conf/hue.ini*
 * */etc/security/keytab/hue.service.keytab* ownership changed (only when security is enabled, the path can be changed by *keytab_hue* parameter)
 * */etc/grid-security/hostcert.pem* copied to */etc/hue/conf/* (only with *https*)
 * */etc/grid-security/hostkey.pem* copied to */etc/hue/conf/* (only with *https*)
* Packages: *hue*
* Services: *hue*
* Users and groups:
 * *hue::hdfs* and *hue::user* classes creates the user *hue* and group *hue*

### Setup Requirements

* Hadoop cluster with WebHDFS or httpfs (httpfs is required for HDFS HA)
* HBase with thrift server (optional)
* Hive Server2 (optional)
* Authorizations set ('hue' user must be enabled in *security.client.protocol.acl*, default '*' is OK)
* Proxy users set for Hadoop core, Hadoop httpfs, Ozzie server
 * for *cesnet::hadoop* puppet module: parameters *hue\_hostnames* and *httpfs\_hostnames*
 * for *cesnet::oozie* puppet module: parameter *hue\_hostnames*

## Usage

### Basic cluster usage

    $master_hostname = 'hdfs.example.com'
    $hue_hostname = 'hue.example.com'
    $secret = 'I sleep with my cats.'

    class { '::hadoop':
      ...
      hue_hostnames => ['hue.example.com'],
    }

    node 'hdfs.example.com' {
      include ::hadoop::namenode
      ...
      include ::hue::hdfs
    }

    node 'hue.example.com' {
      class { '::hue':
        hdfs_hostname => $master_hostname,
        #yarn_hostname  => ...,
        #oozie_hostname => ...,
        secret        => $secret,
      }
    }

### High availability cluster usage

    $cluster_name = 'cluster',
    $master_hostnames = [
      'master1.example.com',
      'master2.example.com',
    ]
    $hue_hostname = 'hue.example.com'
    $secret = "Trump's <PASSWORD> Drumph."

    class { '::hadoop':
      ...
      cluster_name   => $cluster_name,
      hdfs_hostname  => $master_hostnames[0],
      hdfs_hostnames => $master_hostnames[1],
      hue_hostnames  => ['hue.example.com'],
    }

    node 'master1.example.com' {
      include ::hadoop::namenode
      ...
      include ::hue::hdfs
    }

    node 'master2.example.com' {
      include ::hadoop::namenode
      ...
      include ::hue::user
    }

    node 'hue.example.com' {
      class { '::hue':
        defaultFS       => "hdfs://${cluster_name}",
        httpfs_hostname => $hue_hostname,
        #yarn_hostname  => ...,
        #yarn_hostname2 => ...,
        #oozie_hostname => ...,
        secret          => $secret,
      }
    }

There is also needed class *hue::hdfs* on all HDFS Namenodes to authorization work properly. You can use *hue::user* instead, or install *hue-common* package.

It is recommended to set properties *hadoop.yarn\_clusters.default.logical\_name* and *hadoop.yarn\_clusters.ha.logical\_name* according to the *yarn.resourcemanager.ha.rm-ids* from Hadoop YARN. *cesnet-hue* module uses 'rm1' and 'rm2' values, which is *cesnet-hadoop* puppet module default.

### Enable security

Useful parameters: [*https*](#https) [*https\_cachain*](#https_cachain) [*https\_certificate*](#https_certificate) [*https\_private\_key*](#https_private_key) [*https\_passphrase*](#https_passphrase) [*keytab\_hue*](#keytab_hue) [*realm*](#realm)
{'fraction_non_alphanumeric': 0.14217904846009494, 'fraction_numerical': 0.0035323987195054643, 'mean_word_length': 3.4564682734874568, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 32, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1302654', 'n_tokens_mistral': 2989, 'n_tokens_neox': 2866, 'n_words': 752}