<commit_before>---
layout: post
title: "Stax"
subtitle: "Create and manage CloudFormation stacks in AWS"
header-img: "img/mon-field_rows.jpg"
author: <NAME>
tags: [open source, aws, stax, cloudformation]
---

When we were looking to build out an infrastructure in AWS ([Amazon Web Services](https://aws.amazon.com)), we took some time to understand how the [aws-cli](https://github.com/aws/aws-cli) (Universal Command Line Interface for Amazon Web Services) worked. With ideas from other projects found on [GitHub](https://github.com), we started wrapping commands in simple bash scripts, and pointed it to Cloudformation templates written in json. From there [stax](https://github.com/monsantoco/stax) was born, and we have released it as open source under the [BSD 3-clause license](http://opensource.org/licenses/BSD-3-Clause).

It’s easy to get started with stax to create and manage CloudFormation stacks (aka stax) in AWS ([Amazon Web Services](https://aws.amazon.com)) and the project provides several CloudFormation templates, better yet, more can be generated using our [cloudformation-template-generator](https://github.com/MonsantoCo/cloudformation-template-generator).

<!--more-->

For launching new AWS instances with stax, you can use Linux (Debian GNU/Linux 7 and Ubuntu 14.04 are supported) and Apple OS X (tested on 10.10 and 10.9). When running with a default template, you will end up with a VPC setup like the following:

![AWS Stax Diagram](/img/aws-stax.png)

> **NOTICE** as you can see, stax creates an AWS environment from scratch, so it needs to be run by an IAM user that has a role with permissions to create all AWS resources, including IAM roles. You should understand the potential security potential if your AWS environment is compromised. For specific details of what the IAM user can create, you can see the details in the [cloudformation](https://github.com/MonsantoCo/stax/blob/develop/template/vpc-default.json) file.

CoreOS hosts run Docker containers on the Gateway and Service layer, while [Amazon Linux](https://aws.amazon.com/amazon-linux-ami/) instances serve as NAT devices to direct traffic for the internal hosts. Additionally, one of the biggest requests we had was to simplify the connectivity to nodes once the stax was created. For this we have the `stax connect` function, that will ssh through the bastion/ssh host, and log you on to either a host on the gateway layer, or the services layer.

The Gateway layer:

<pre>
$ stax connect gateway
     _
    | | every cloud has a silicon lining
 ___| |_ __ ___  __
/ __| __/ _` \ \/ /
\__ \ || (_| |>  <
|___/\__\__,_/_/\_\  0.9

[ ---- ]  checking if stax build is complete
[ ---- ]  describe stax
[ NAME ]  vpc-stax-42179-unrepulsed
[ ---- ]    querying aws
[ ---- ]    query complete
[ ---- ]    see /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.json for details
[ ---- ]    stack vpc-stax-42179-unrepulsed build complete
[ ---- ]  connecting to stax: gateway
Last login: Tue Apr 28 18:57:07 2015 from 10.183.1.208
CoreOS stable (633.1.0)
Failed Units: 3
  nd.service
  nginx.service
  router.service
core@ip-10-183-0-131 ~ $
</pre>

So you’re logged into a gateway CoreOS box, and from the current `/etc/motd` you’ll see that in this case some services has failed to start.

Now we’ll login to the Service layer:

<pre>
$ stax connect service
     _
    | | all glory to the hypnotoad!
 ___| |_ __ ___  __
/ __| __/ _` \ \/ /
\__ \ || (_| |>  <
|___/\__\__,_/_/\_\  0.9

[ ---- ]  checking if stax build is complete
[ ---- ]  describe stax
[ NAME ]  vpc-stax-42179-unrepulsed
[ ---- ]    querying aws
[ ---- ]    query complete
[ ---- ]    see /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.json for details
[ ---- ]    stack vpc-stax-42179-unrepulsed build complete
[ ---- ]  getting public IP (EIP)
[ ---- ]    public IP (EIP): 192.168.3.11
[ ---- ]    writing to /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.bastion
[ ---- ]  get gateway and service IPs
[ ---- ]    querying aws for gateway hosts
[ ---- ]    getting service leaders
[ ---- ]    querying aws for service hosts
[ ---- ]    getting elk host
[ ---- ]  creating ssh_config
[ ---- ]    writing /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.ssh_config
[ ---- ]    created /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.ssh_config
[ ---- ]  connecting to stax: service
CoreOS stable (633.1.0)
core@ip-10-183-0-111 ~ $
</pre>

Keeping in mind that this is only available when you’re in development, no external access is allowed to internal nodes directly without the bastion/ssh host, and shutting it down when running in production is highly recommended.

Lastly a quick run of stax without arguments will show you the usage, giving you an idea of all lf the tasks the tool is capable of:

<pre>
$ ./stax
Usage: stax [OPTIONS] COMMAND

Options:
  -c,--config=CONFIG       Use file CONFIG rather than config/vpc-default.json
  -d,--debug               Turn on verbose messages
  -h,--help                Output this message
  -j,--jump=IP             SSH through host with IP address IP
  -m,--module=MOD          Use config/MOD.json and template/MOD.json
  -t,--template=TEMPLATE   Use file TEMPLATE rather than template/vpc-default.json
  -v,--version             Print name and version information
  -y,--yes                 Do not prompt for confirmation

If an argument is required for a long option, so to the short. Same for
optional arguments.

Commands:
  add                Add functionality to an existing VPC
  check              Run various tests against an existing stax
  connect [TARGET]   Connect to bastion|gateway|service in the VPC stax over SSH
  create             Create a new VPC stax in AWS
  describe           Describe the stax created from this host
  delete             Delete the existing VPC stax
  dockerip-update    Fetch docker IP addresses and update related files
  fleet              Run various fleetctl commands against the fleet cluster
  help               Output this message
  history            View history of recently created/deleted stax
  list               List all completely built and running stax
  rds PASSWORD       Create an RDS instance in the DB subnet
  rds-delete RDSIN   Delete RDS instance RDSIN
  remove ADD         Remove the previously added ADD
  services           List servers that are available to run across a stax
  start SERVICE      Start service SERVICE in the fleet cluster
  test               Automated test to exercise functionality of stax
  validate           Validate CloudFormation template

For more help, check the docs: https://github.com/MonsantoCo/stax
</pre>

To try it out, just clone the repo and follow the documentation in the readme.

<pre>
git clone https://github.com/MonsantoCo/stax
more README.md
</pre>

We hope that this tool will be useful to the community and look forward to feedback on what people think of it. We’re open to issues you might create or pull requests to make stax better.<commit_msg>2015-06-24-stax.md: updated and cleaned up a bit - added github, twitter, and avatar for Phil - remove reference to private repos - cleaned up screen captures to remove obsolete references, fix command   line, and ensure proper order - changed pre to markdown code<commit_after>---
layout: post
title: "Stax"
subtitle: "Create and manage CloudFormation stacks in AWS"
header-img: "img/mon-field_rows.jpg"
author: <NAME>
githubProfile : "philcryer"
twitterHandle : "fak3r"
avatarUrl : "https://avatars0.githubusercontent.com/u/43070?v=3"
tags: [open source, aws, stax, cloudformation]
---

When we were looking to build out an infrastructure in AWS ([Amazon Web Services](https://aws.amazon.com)), we took some time to understand how the [aws-cli](https://github.com/aws/aws-cli) (Universal Command Line Interface for Amazon Web Services) worked. With ideas from other projects found on [GitHub](https://github.com), we started wrapping commands in simple BASH scripts, and pointed it to CloudFormation templates written in JSON. From there [stax](https://github.com/monsantoco/stax) was born, and we have released it as open source under the [BSD 3-clause license](http://opensource.org/licenses/BSD-3-Clause).

It’s easy to get started with stax to create and manage CloudFormation stacks (aka stax) in AWS ([Amazon Web Services](https://aws.amazon.com)) and the project provides several CloudFormation templates, or you can use your own.

For launching new AWS instances with stax, you can use Linux (Debian GNU/Linux 7 and Ubuntu 14.04 are supported) and Apple OS X (tested on 10.10 and 10.9). When running with a default template, you will end up with a VPC setup like the following:

![AWS Stax Diagram](/img/aws-stax.png)

> **NOTICE** as you can see, stax creates an AWS environment from scratch, so it needs to be run by an IAM user that has a role with permissions to create all AWS resources, including IAM roles. You should understand the potential security potential if your AWS environment is compromised. For specific details of what the IAM user can create, you can see the details in the [cloudformation](https://github.com/MonsantoCo/stax/blob/develop/template/vpc-default.json) file.

CoreOS hosts run Docker containers on the Gateway and Service layer, while [Amazon Linux](https://aws.amazon.com/amazon-linux-ami/) instances serve as NAT devices to direct traffic for the internal hosts. Additionally, one of the biggest requests we had was to simplify the connectivity to nodes once the stax was created. For this we have the `stax connect` function, that will ssh through the bastion/ssh host, and log you on to either a host on the gateway layer, or the services layer.

The Gateway layer:

```
$ ./stax connect gateway
     _
    | | every cloud has a silicon lining
 ___| |_ __ ___  __
/ __| __/ _` \ \/ /
\__ \ || (_| |>  <
|___/\__\__,_/_/\_\  0.9

[ ---- ]  checking if stax build is complete
[ ---- ]  describe stax
[ NAME ]  vpc-stax-42179-unrepulsed
[ ---- ]    querying aws
[ ---- ]    query complete
[ ---- ]    see /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.json for details
[ ---- ]    stack vpc-stax-42179-unrepulsed build complete
[ ---- ]  getting public IP (EIP)
[ ---- ]    public IP (EIP): 192.168.3.11
[ ---- ]    writing to /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.bastion
[ ---- ]  get gateway and service IPs
[ ---- ]    querying aws for gateway hosts
[ ---- ]    getting service leaders
[ ---- ]    querying aws for service hosts
[ ---- ]  creating ssh_config
[ ---- ]    writing /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.ssh_config
[ ---- ]    created /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.ssh_config
[ ---- ]  connecting to stax: gateway
Last login: Tue Apr 28 18:57:07 2015 from 10.183.1.208
CoreOS stable (633.1.0)
core@ip-10-183-0-131 ~ $
```

So you’re logged into a gateway CoreOS box, and from the current `/etc/motd` you can see what version of CoreOS you are running.

Now we’ll login to the Service layer:

```
$ ./stax connect service
     _
    | | all glory to the hypnotoad!
 ___| |_ __ ___  __
/ __| __/ _` \ \/ /
\__ \ || (_| |>  <
|___/\__\__,_/_/\_\  0.9

[ ---- ]  checking if stax build is complete
[ ---- ]  describe stax
[ NAME ]  vpc-stax-42179-unrepulsed
[ ---- ]    querying aws
[ ---- ]    query complete
[ ---- ]    see /home/phil/devel/stax/run/vpc-stax-42179-unrepulsed.json for details
[ ---- ]    stack vpc-stax-42179-unrepulsed build complete
[ ---- ]  connecting to stax: service
CoreOS stable (633.1.0)
core@ip-10-183-0-111 ~ $
```

Keeping in mind that this is only available when you’re in development, no external access is allowed to internal nodes directly without the bastion/ssh host, and shutting it down when running in production is highly recommended.

Lastly a quick run of stax without arguments will show you the usage, giving you an idea of all lf the tasks the tool is capable of:

```
$ ./stax
Usage: stax [OPTIONS] COMMAND

Options:
  -c,--config=CONFIG       Use file CONFIG rather than config/vpc-default.json
  -d,--debug               Turn on verbose messages
  -h,--help                Output this message
  -j,--jump=IP             SSH through host with IP address IP
  -m,--module=MOD          Use config/MOD.json and template/MOD.json
  -t,--template=TEMPLATE   Use file TEMPLATE rather than template/vpc-default.json
  -v,--version             Print name and version information
  -y,--yes                 Do not prompt for confirmation

If an argument is required for a long option, so to the short. Same for
optional arguments.

Commands:
  add                Add functionality to an existing VPC
  check              Run various tests against an existing stax
  connect [TARGET]   Connect to bastion|gateway|service in the VPC stax over SSH
  create             Create a new VPC stax in AWS
  describe           Describe the stax created from this host
  delete             Delete the existing VPC stax
  dockerip-update    Fetch docker IP addresses and update related files
  fleet              Run various fleetctl commands against the fleet cluster
  help               Output this message
  history            View history of recently created/deleted stax
  list               List all completely built and running stax
  rds PASSWORD       Create an RDS instance in the DB subnet
  rds-delete RDSIN   Delete RDS instance RDSIN
  remove ADD         Remove the previously added ADD
  services           List servers that are available to run across a stax
  start SERVICE      Start service SERVICE in the fleet cluster
  test               Automated test to exercise functionality of stax
  validate           Validate CloudFormation template

For more help, check the docs: https://github.com/MonsantoCo/stax
```

To try it out, just clone the repo and follow the documentation in the readme.

```
$ git clone https://github.com/MonsantoCo/stax
$ more README.md
```

We hope that this tool will be useful to the community and look forward to feedback on what people think of it. We’re open to issues you might create or pull requests to make stax better.
{'fraction_non_alphanumeric': 0.09284853639520874, 'fraction_numerical': 0.018357077043022186, 'mean_word_length': 2.9590347923681257, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 20, 'https://': 20, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '781032', 'n_tokens_mistral': 4382, 'n_tokens_neox': 4044, 'n_words': 1702}