<commit_before># Overview

Create a 3 node Marathon / Mesos cluster with a single master node and 2 slave nodes. Two environments are currently supported.

* Vagrant    - 3 local VMs running CoreOS on VirtualBox or VMware Fusion.
* Packet.net - 3 physical servers running CoreOS. You need an account with [Packet.net](https://packet.net) to do this.

## WORK IN PROGRESS

Vagrant setup is fully working. On Packet there is a problem with Consul DNS not starting reliably.

## Vagrant Setup

1) Install dependencies

* [VirtualBox](https://www.virtualbox.org/) 4.3.10 or greater.
* [Vagrant](https://www.vagrantup.com/downloads.html) 1.6 or greater.

2) Clone this project.

```
$ git clone https://github.com/force12/coreos-marathon/
$ cd coreos-marathon
```

3) Startup and SSH

Boxes are named core-01..03.

```
$ vagrant up
$ vagrant ssh core-01
```

4) Access web UIs

Once all the Docker images have been downloaded and started the web UIs will be availabled at.
Note: This can take up to 10 minutes and the images themselves are large.

* Marathon: http://172.17.8.101:8080
* Mesos: http://172.17.8.101:5050

## Vagrant Teardown

1) Delete Vagrant VMs.

```
vagrant destroy
```

## Packet Setup

1) Register for an account with [Packet](https://packet.net).

2) Clone this project.

```
$ git clone https://github.com/force12/coreos-marathon/
$ cd coreos-marathon
```

3) Set API key and config.

```
# .env.packet

API_TOKEN=YOUR_API_KEY
PROJECT_NAME=example
DEVICE_PREFIX=core
PLAN=baremetal_1
# Parsippany, NJ
FACILITY=ewr1
# Either coreos_stable or coreos_beta
OPERATING_SYSTEM=coreos_beta
```

4) Start cluster of 3 physical servers running CoreOS beta.

```
$ cp .env.packet .env

$ bundle install
$ bundle exec rake packet:cluster:create

Cluster is being provisioned
```

5) Check status of the console in the Packet web UI. Once all 3 servers are provisioned
get the public IP address of the core-01 node. The Marathon web UI is available on port 8080 and Mesos web UI is on port 5050.

## Packet Teardown

1) Shutdown server cluster.

```
$ bundle exec rake packet:cluster:destroy

Cluster has been shutdown
```

## Components

| Name        | Role                  | Version
| ------------|-----------------------|-------------
| CoreOS      | Operating System      | beta channel
| Consul      | Service Discovery     | 0.4
| Registrator | Service Registration  | v5
| Zookeeper   | Service Discovery     | 3.4.6
| Mesos       | Resource Pooling      | 0.20.1
| Marathon    | Resource Scheduling   | 0.7.5

### Bootstrapping

* CoreOS cloud-config starts the etcd2 and fleet services.
* A new discovery token is retrieved from discovery.etcd.io each time and used to form a cluster.
* A 3 node Consul cluster is bootstrapped using etcd2.
* Zookeeper is registered with Consul and accessed via the Consul DNS interface.
* Mesos and Marathon use Zookeeper for service discovery.
* Consul is needed because on Packet the IP address of the master node running Zookeeper is dynamic.

## TODO

* Secure web UIs with SSL certificate and HTTP basic authentication.
* Fix Consul DNS issue with fully qualified hostnames e.g. core-01.force12.io

## Thanks

Thanks to [Packet](https://www.packet.net/) for the great support while we automated the cluster setup.

This template is based upon these projects.

* [coreos-vagrant](https://github.com/coreos/coreos-vagrant) - Vagrant template and boxes provided by CoreOS.
* [consul-coreos](https://github.com/democracyworks/consul-coreos) - bootstraps a Consul cluster using etcd2.<commit_msg>Update README with new bootstrapping process.<commit_after># Overview

Create a 3 node Marathon / Mesos cluster with a single master node and 2 slave nodes. Two environments are currently supported.

* Vagrant    - 3 local VMs running CoreOS on VirtualBox or VMware Fusion.
* Packet.net - 3 physical servers running CoreOS. You need an account with [Packet.net](https://packet.net) to do this.

## Vagrant Setup

1) Install dependencies

* [VirtualBox](https://www.virtualbox.org/) 4.3.10 or greater.
* [Vagrant](https://www.vagrantup.com/downloads.html) 1.6 or greater.

2) Clone this project.

```
$ git clone https://github.com/force12/coreos-marathon/
$ cd coreos-marathon
```

3) Startup and SSH

Boxes are named core-01..03.

```
$ vagrant up
$ vagrant ssh core-01
```

4) Access web UIs

Once all the Docker images have been downloaded and started the web UIs will be availabled at.
Note: This can take up to 10 minutes and the images themselves are large.

* Marathon: http://172.17.8.101:8080
* Mesos: http://172.17.8.101:5050

## Vagrant Teardown

1) Delete Vagrant VMs.

```
vagrant destroy
```

## Packet Setup

1) Register for an account with [Packet](https://packet.net).

2) Clone this project.

```
$ git clone https://github.com/force12/coreos-marathon/
$ cd coreos-marathon
```

3) Set API key and config.

```
# .env.packet

API_TOKEN=YOUR_API_KEY
PROJECT_NAME=example
DEVICE_PREFIX=core
PLAN=baremetal_1
# Parsippany, NJ
FACILITY=ewr1
# Either coreos_beta or coreos_alpha
OPERATING_SYSTEM=coreos_beta
```

4) Start cluster of 3 physical servers running CoreOS beta.

```
$ cp .env.packet .env

$ bundle install
$ bundle exec rake packet:cluster:create

Cluster is being provisioned
```

5) Check status of the console in the Packet web UI. Once all 3 servers are provisioned
get the public IP address of the core-01 node. The Marathon web UI is available on port 8080 and Mesos web UI is on port 5050.

## Packet Teardown

1) Shutdown server cluster.

```
$ bundle exec rake packet:cluster:destroy

Cluster has been shutdown
```

### Bootstrapping

* A fresh discovery token is retrieved from discovery.etcd.io for each cluster.
* CoreOS cloud-config starts etcd2 and fleet and a oneshot unit that starts the cluster.
* A 3 node Consul cluster is bootstrapped using the etcd cluster.
* Zookeeper is registered with Consul and accessed via the Consul DNS interface.
* Mesos and Marathon use Zookeeper for service discovery.

## Docker Image

The oneshot unit launches a Docker container using the image [quay.io/rossf7/coreos-marathon:latest](https://quay.io/repository/rossf7/coreos-marathon).
This image is based on Alpine Linux and installs Ruby and the fleetctl binary. The container runs a Ruby script that starts the cluster using fleet.

## Thanks

Thanks to [Packet](https://www.packet.net/) for the great support while we automated the cluster setup.

This template is based upon these projects.

* [coreos-vagrant](https://github.com/coreos/coreos-vagrant) - Vagrant template and boxes provided by CoreOS.
* [consul-coreos](https://github.com/democracyworks/consul-coreos) - bootstraps a Consul cluster using etcd2.
{'fraction_non_alphanumeric': 0.09715994020926756, 'fraction_numerical': 0.02571001494768311, 'mean_word_length': 3.7656695156695155, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 19, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2418757', 'n_tokens_mistral': 2213, 'n_tokens_neox': 2027, 'n_words': 809}