<commit_before># Bamboo  [![Build Status](https://travis-ci.org/QubitProducts/bamboo.svg?branch=master)](https://travis-ci.org/QubitProducts/bamboo)

![bamboo-logo](https://cloud.githubusercontent.com/assets/37033/4110258/a8cc58bc-31ef-11e4-87c9-dd20bd2468c2.png)

Bamboo is a web daemon that automatically configures
HAProxy for web services deployed on [Apache Mesos](http://mesos.apache.org) and [Marathon](https://mesosphere.github.io/marathon/).

It features:

* User interface for configuring DNS mapping to Marathon ID
* Rest API for configuring DNS mapping; when application deployment in Marathon is automated and work with other DNS managing services, API is handy
* Auto configure HAProxy configuration file based your template; you can provision your own template in production to enable ssl and HAProxy stats interface, or configuring different load balance strategy
* Optionally hanldes health check endpoint if Marathon app is configured with [Healthchecks](https://mesosphere.github.io/marathon/docs/health-checks.html)
* Daemon itself is stateless; enables horizontal replication and scalability
* Developed in Golang, deployment on HAProxy instance has no additional dependency
* Optionally integrates with StatsD to monitor configuration reload event  


### Compatibility and next release:

We tested on Marathon 0.6.1 and Mesos 0.19 on production. 

There will be a release soon to support Marathon 0.7 with a lot of feature and performance improvements, including support mixed non-DNS, DNS and custom HAProxy frontend acl rules. Be prepared! 


![user-interface](https://cloud.githubusercontent.com/assets/37033/4110199/a6226b8e-31ee-11e4-9734-68e0da00767c.png)

## User Interface

If you have very small scale web services to manage, UI is useful to manage and visualize current state of DNS mapping.
You can find out if DNS is assigned or missing from the interface. Of course, you can configure HAProxy template to load balance Bamboo. 

![user-interface](https://cloud.githubusercontent.com/assets/37033/4109769/318f2ad2-31e9-11e4-8f5f-b6a3368412b9.png)

## StatsD Monitoring

![bamboo-graphite](https://cloud.githubusercontent.com/assets/37033/4117219/cef5cea2-328e-11e4-8346-ecc4e4e6046b.png)

## Configuration and Template

Bamboo binary accepts `-config` option to specify application configuration JSON file location. Type `-help` to get current available options.

Example configuration and HAProxy template can be found under [config/production.example.json](config/production.example.json) and  [config/haproxy_template.cfg](config/haproxy_template.cfg)
This section tries to explain usage in code comment style:

```JavaScript

{
  // Marathon instance configuration
  "Marathon": {
    // Marathon service HTTP endpoint
    "Endpoint": "http://localhost:8080",
    // Same configuration as Marathon Zookeeper
    "Zookeeper": {
      "Host": "zk01.example.com:2812,zk02.example.com:2812",
      // Marathon Zookeeper state  
      // Marathon default set to /marathon/state
      "Path": "/marathon/state",
      // Number of seconds to delay the reload event
      "ReportingDelay": 5
    }
  },
   
  // DNS mapping information is stored in Zookeeper
  // Bamboo will create this path if it does not already exist
  "DomainMapping": {
    "Zookeeper": {
      // Use the same ZK setting if you run on the same ZK cluster
      "Host": "zk01.example.com:2812,zk02.example.com:2812",
      "Path": "/marathon-haproxy/state",
      "ReportingDelay": 5
    }
  },
  
  // Make sure using absolute path on production
  "HAProxy": {
    "TemplatePath": "/var/bamboo/haproxy_template.cfg",
    "OutputPath": "/etc/haproxy/haproxy.cfg",
    "ReloadCommand": "haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)"
  },
  
  // Enable or disable StatsD event tracking
  "StatsD": {
    "Enabled": false,
    // StatsD or Graphite server host
    "Host": "localhost:8125",
    // StatsD namespace prefix
    // If you have multiple Bamboo instances, you might want to label each node
    // by bamboo-server.production.n1.
    "Prefix": "bamboo-server.production."
  }
}
```

### Environment Variables

Configuration in the `production.json` file can be overridden with environment variables below. This is generally useful when you are building a Docker image for Bamboo and HAProxy. If they are not specified then the values from the configuration file will be used.

Environment Variable | Corresponds To
---------------------|---------------
`MARATHON_ZK_HOST` | Marathon.Zookeeper.Host
`MARATHON_ZK_PATH` | Marathon.Zookeeper.Path
`MARATHON_ENDPOINT` | Marathon.Endpoint
`DOMAIN_ZK_HOST` | DomainMapping.Zookeeper.Host
`DOMAIN_ZK_PATH` | DomainMapping.Zookeeper.Path
`HAPROXY_TEMPLATE_PATH` | HAProxy.TemplatePath
`HAPROXY_OUTPUT_PATH` | HAProxy.OutputPath
`HAPROXY_RELOAD_CMD` | HAProxy.ReloadCommand


## REST APIs

POST /api/state/domains

Creates mapping from Marathon application ID to a DNS

```bash
curl -i -X POST -d '{"id":"app-1","value":"app1.example.com"}' http://localhost:8000/api/state/domains
```


PUT /api/state/domains/:id

Updates mapping of an existing Marathon application ID to a new DNS

```bash
curl -i -X PUT -d '{"id":"app-1","value":"app1-beta.example.com"}' http://localhost:8000/api/state/domains/app-1
```


DELETE /api/state/domains/:id

Deletes mapping of an existing Marathon ID DNS mapping

```bash
curl -i -X DELETE http://localhost:8000/api/state/domains/app-1
```

GET /status

Bamboo webapp's healthcheck point

```
curl -i http://localhost:8000/status
```


## Deployment

We recommend installing binary with deb or rpm package. 
The repository includes examples of [a Jenkins build script](https://github.com/QubitProducts/bamboo/blob/master/builder/ci-jenkins.sh)
and [a deb packages build script](https://github.com/QubitProducts/bamboo/blob/master/builder/build.sh).
Read comments in the script to customize your build distribution workflow.

In short, [install fpm](https://github.com/jordansissel/fpm) and run the following command:

```
go build bamboo.go
./builder/build.sh
```

A deb package will be generated in `./builder` directory. You can copy to a server or publish to your own apt repository.

The example deb package deploys:

* Upstart job [`bamboo-server`](https://github.com/QubitProducts/bamboo/blob/master/builder/bamboo-server), e.g. upstart assumes `/var/bamboo/production.json` is configured correctly.
* Application directory is under `/opt/bamboo/`
* Configuration and logs is under `/var/bamboo/`
* Log file is rotated automatically

### As a Docker container

There is a `Dockerfile` that will allow Bamboo to be built and run from within a Docker container.

#### Building the image

The Docker image can be built and added to your local repository with the following command from within the project root directory:

````
docker build -t bamboo .
````

#### Running Bamboo as a Docker container

Once the image has been built, running as a container is straightforward - you do however still need to provide the configuration to the image as environment variables. Docker allows two options for this - using the `-e` option  or by putting them in a file and using the `--env-file` option. For this example we will use the former and we will map through ports 8000 and 80 to the docker host (obviously the hosts configured here will need to be reachable from this container):

````
docker run -t -i --rm -p 8000:8000 -p 80:80 \
    -e MARATHON_ZK_HOST=zk:2181 \
    -e MARATHON_ZK_PATH=/marathon \
    -e MARATHON_ENDPOINT=http://marathon:8080 \
    -e DOMAIN_ZK_HOST=zk:2181 \
    -e DOMAIN_ZK_PATH=/bamboo \
    bamboo -bind=":8000"
````

## Development and Contribution
<commit_msg>Added new documentation for 0.2.0 release<commit_after># Bamboo  [![Build Status](https://travis-ci.org/QubitProducts/bamboo.svg?branch=master)](https://travis-ci.org/QubitProducts/bamboo)

![bamboo-logo](https://cloud.githubusercontent.com/assets/37033/4110258/a8cc58bc-31ef-11e4-87c9-dd20bd2468c2.png)

Bamboo is a web daemon that automatically configures
HAProxy for web services deployed on [Apache Mesos](http://mesos.apache.org) and [Marathon](https://mesosphere.github.io/marathon/).

It features:

* User interface for configuring HAProxy ACL rules for each Marathon application
* Rest API for configuring proxy ACL rules
* Auto configure HAProxy configuration file based your template; you can provision your own template in production to enable SSL and HAProxy stats interface, or configuring different load balance strategy
* Optionally hanldes health check endpoint if Marathon app is configured with [Healthchecks](https://mesosphere.github.io/marathon/docs/health-checks.html)
* Daemon itself is stateless; enables horizontal replication and scalability
* Developed in Golang, deployment on HAProxy instance has no additional dependency
* Optionally integrates with StatsD to monitor configuration reload event


### Compatibility

Bamboo v0.1.1 supports Marathon 0.6 and Mesos 0.19.x

Bamboo v0.2.0 supports Marathon 0.7 (with [http_callback enabled](https://mesosphere.github.io/marathon/docs/rest-api.html#event-subscriptions)) and Mesos 0.20.x. Since v0.2.0, Bamboo supports both DNS and non-DNS proxy ACL rules.


## Deployment Guide

You can deploy Bamboo with HAProxy on each Mesos slave. Each web service being allocated on Mesos Slave can discover services via localhost or domain you assigned by ACL rules. Alternatively, you can deploy Bamboo and HAProxy on separate instances, which means you need to loadbalance HAProxy cluster.

![bamboo-setup-guide](https://cloud.githubusercontent.com/assets/37033/4110199/a6226b8e-31ee-11e4-9734-68e0da00767c.png)

## User Interface

UI is useful to manage and visualize current state of proxy rules. Of course, you can configure HAProxy template to load balance Bamboo.

![user-interface-list](https://cloud.githubusercontent.com/assets/37033/4320901/527988dc-3f3b-11e4-8672-666605eb1ddf.png)

![user interface](https://cloud.githubusercontent.com/assets/37033/4320873/2ac65c48-3f3b-11e4-969e-52381dd33aae.png)

## StatsD Monitoring

![bamboo-graphite](https://cloud.githubusercontent.com/assets/37033/4117219/cef5cea2-328e-11e4-8346-ecc4e4e6046b.png)

## Configuration and Template

Bamboo binary accepts `-config` option to specify application configuration JSON file location. Type `-help` to get current available options.

Example configuration and HAProxy template can be found under [config/production.example.json](config/production.example.json) and  [config/haproxy_template.cfg](config/haproxy_template.cfg)
This section tries to explain usage in code comment style:

```JavaScript

{
  // Marathon instance configuration
  "Marathon": {
    // Marathon service HTTP endpoint
    "Endpoint": "http://localhost:8080",
    // Same configuration as Marathon Zookeeper
    "Zookeeper": {
      "Host": "zk01.example.com:2812,zk02.example.com:2812",
      // Marathon Zookeeper state  
      // Marathon default set to /marathon/state
      "Path": "/marathon/state",
      // Number of seconds to delay the reload event
      "ReportingDelay": 5
    }
  },

  "Bamboo": {

    // Bamboo's HTTP address can be accessed by Marathon
    // This is used for Marathon HTTP callback
    "Host": "http://localhost:8000",

    // Proxy setting information is stored in Zookeeper
    // Bamboo will create this path if it does not already exist
    "Zookeeper": {
      // Use the same ZK setting if you run on the same ZK cluster
      "Host": "zk01.example.com:2812,zk02.example.com:2812",
      "Path": "/marathon-haproxy/state",
      "ReportingDelay": 5
    }
  }
  
  
  // Make sure using absolute path on production
  "HAProxy": {
    "TemplatePath": "/var/bamboo/haproxy_template.cfg",
    "OutputPath": "/etc/haproxy/haproxy.cfg",
    "ReloadCommand": "haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)"
  },
  
  // Enable or disable StatsD event tracking
  "StatsD": {
    "Enabled": false,
    // StatsD or Graphite server host
    "Host": "localhost:8125",
    // StatsD namespace prefix
    // If you have multiple Bamboo instances, you might want to label each node
    // by bamboo-server.production.n1.
    "Prefix": "bamboo-server.production."
  }
}
```

### Environment Variables

Configuration in the `production.json` file can be overridden with environment variables below. This is generally useful when you are building a Docker image for Bamboo and HAProxy. If they are not specified then the values from the configuration file will be used.

Environment Variable | Corresponds To
---------------------|---------------
`MARATHON_ZK_HOST` | Marathon.Zookeeper.Host
`MARATHON_ZK_PATH` | Marathon.Zookeeper.Path
`MARATHON_ENDPOINT` | Marathon.Endpoint
`BAMBOO_HOST` | Bamboo.Host
`BAMBOO_ZK_HOST` | Bamboo.Zookeeper.Host
`BAMBOO_ZK_PATH` | Bamboo.Zookeeper.Path
`HAPROXY_TEMPLATE_PATH` | HAProxy.TemplatePath
`HAPROXY_OUTPUT_PATH` | HAProxy.OutputPath
`HAPROXY_RELOAD_CMD` | HAProxy.ReloadCommand


## REST APIs


#### GET /api/state

Shows the data structure used for rendering template

```bash
curl -i http://localhost:8000/api/state
```

#### POST /api/services

Creates a service configuration for a Marathon application ID

```bash
curl -i -X POST -d '{"id":"/app-1","acl":"hdr(host) -i app-1.example.com"}' http://localhost:8000/api/services
```

#### PUT /api/services/:id

Updates an existing service configuraiton for a Marathon application. `:id` is  URI encoded Marathon application ID

```bash
curl -i -X PUT -d '{"id":"/app-1", "acl":"path_beg -i /group/app-1"}' http://localhost:8000/api/services/%2Fapp-1
```


#### DELETE /api/services/:id

Deletes an existing service configuration. `:id` is  URI encoded Marathon application ID

```bash
curl -i -X DELETE http://localhost:8000/api/services/%2Fapp-1
```

#### GET /status

Bamboo webapp's healthcheck point

```
curl -i http://localhost:8000/status
```


## Deployment

We recommend installing binary with deb or rpm package. 
The repository includes examples of [a Jenkins build script](https://github.com/QubitProducts/bamboo/blob/master/builder/ci-jenkins.sh)
and [a deb packages build script](https://github.com/QubitProducts/bamboo/blob/master/builder/build.sh).
Read comments in the script to customize your build distribution workflow.

In short, [install fpm](https://github.com/jordansissel/fpm) and run the following command:

```
go build bamboo.go
./builder/build.sh
```

A deb package will be generated in `./builder` directory. You can copy to a server or publish to your own apt repository.

The example deb package deploys:

* Upstart job [`bamboo-server`](https://github.com/QubitProducts/bamboo/blob/master/builder/bamboo-server), e.g. upstart assumes `/var/bamboo/production.json` is configured correctly.
* Application directory is under `/opt/bamboo/`
* Configuration and logs is under `/var/bamboo/`
* Log file is rotated automatically

### As a Docker container

There is a `Dockerfile` that will allow Bamboo to be built and run from within a Docker container.

#### Building the image

The Docker image can be built and added to your local repository with the following command from within the project root directory:

````
docker build -t bamboo .
````

#### Running Bamboo as a Docker container

Once the image has been built, running as a container is straightforward - you do however still need to provide the configuration to the image as environment variables. Docker allows two options for this - using the `-e` option  or by putting them in a file and using the `--env-file` option. For this example we will use the former and we will map through ports 8000 and 80 to the docker host (obviously the hosts configured here will need to be reachable from this container):

````
docker run -t -i --rm -p 8000:8000 -p 80:80 \
    -e MARATHON_ZK_HOST=zk:2181 \
    -e MARATHON_ZK_PATH=/marathon \
    -e MARATHON_ENDPOINT=http://marathon:8080 \
    -e BAMBOO_HOST=http://bamboo:8000 \
    -e BAMBOO_ZK_HOST=zk:2181 \
    -e BAMBOO_ZK_PATH=/bamboo \
    bamboo -bind=":8000"
````

## Development and Contribution

{'fraction_non_alphanumeric': 0.10717821782178218, 'fraction_numerical': 0.03180693069306931, 'mean_word_length': 4.0503125, 'pattern_counts': {'":': 49, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 26, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4342376', 'n_tokens_mistral': 5420, 'n_tokens_neox': 4835, 'n_words': 1659}