<commit_before>﻿# nubis-dpaste
This is a Nubis deployment of the [dpaste](https://github.com/bartTC/dpaste) application. The purpose of this project is to demonstrate an example of a django / python deployment. This repository is an example of a "deployment repository", in other words a repository that does not contain any application code.

## Repository Structure
The structure of the repository is quite simple. The application is installed as a git submodule. There is a directory called *nubis* which contains all of the nubis modules necessary to deploy the application.


## Deployment Process
Currently there are a few steps necessary to deploy this project. We intend to simplify this process going forward. While these steps are listed in order to build and deploy, it is typically not necessary to run the build steps. This means you can skip the Packer bits and jump straight to the Terraform section.


### Puppet
We are using [puppet](http://puppetlabs.com/) in this example to bootstrap up our VM. Puppet installs and configures services such as *Apache* and *MySql*. We are using the [nubis-puppet](https://github.com/mozilla/nubis-puppet) project for our module collection. This is conveniently installed on the *base image* (built by [nubis-base](https://github.com/mozilla/nubis-base)) that we are going to use as the starting image for our Packer build in the next step.


### Packer
[Packer](https://www.packer.io/) is the piece that will build our AMI. It is made up of a few pieces:

1. The [main.json](https://github.com/mozilla/nubis-dpaste/tree/master/nubis/packer/main.json) file which contains:
    * A *builders* statement describing where to build the AMI
    * One or more *provisioners* statements for calling puppet standalone and any bootstrapping commands for the application
2. A [variables.json](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/packer/variables.json-dist) file which contains things like:
    * AWS credentials
    * base AMI ID

To run packer, from the repository root you first need to create your *variables.json* file, for which there is a template provided (variables.json-dist). After which you simply call packer like so:
```bash
packer build -var-file=nubis/packer/variables.json -var release=0 -var build=1 nubis/packer/main.json
```
This takes around *11m 18.488s* to complete.


### Terraform:
The next step is to take the shiny new AMI that Packer built and deploy it. This is where [Terraform](https://www.terraform.io/) comes into play. Terraform is our infrastructure deployment framework, but not to worry it is really not as complicated as its name implies. It consists of a few files:

1. [inputs.tf](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/inputs.tf) simply lists the variables you might need to provide
2. [main.tf](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/main.tf) is where the real heavy lifting takes place. This is where you describe your infrastructure. Thisgs like EC2 instances, security groups, ELBs and so on.
3. [outputs.tf](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/outputs.tf) describes what information from the build we want to make available (via Consul) for later reference.
4. [terraform.tfvars](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/terraform.tfvars-dist) is where you will set your AWS credentials and such.

To run terraform, from the repository root you first need to create your *terraform.tfvars* file, for which there is a template provided (terraform.tfvars-dist). After which you simply call terraform like so:

To see what resources will be created, destroyed, or refreshed:
```bash
terraform plan -var-file=nubis/terraform/terraform.tfvars nubis/terraform/
```
To apply the plan (do the work)
```bash<commit_msg>Added link and format clean up<commit_after>﻿# nubis-dpaste
This is a Nubis deployment of the [dpaste](https://github.com/bartTC/dpaste) application. The purpose of this project is to demonstrate an example of a django / python deployment. This repository is an example of a "deployment repository", in other words a repository that does not contain any application code.

## Repository Structure
The structure of the repository is quite simple. The application is installed as a git submodule. There is a directory called *nubis* which contains all of the nubis modules necessary to deploy the application.


## Deployment Process
Currently there are a few steps necessary to deploy this project. We intend to simplify this process going forward. While these steps are listed in order to build and deploy, it is typically not necessary to run the build steps. This means you can skip the Packer bits and jump straight to the [Terraform](https://github.com/tinnightcap/nubis-dpaste#terraform) section.


### Puppet
We are using [puppet](http://puppetlabs.com/) in this example to bootstrap up our VM. Puppet installs and configures services such as *Apache* and *MySql*. We are using the [nubis-puppet](https://github.com/mozilla/nubis-puppet) project for our module collection. This is conveniently installed on the *base image* (built by [nubis-base](https://github.com/mozilla/nubis-base)) that we are going to use as the starting image for our Packer build in the next step.


### Packer
[Packer](https://www.packer.io/) is the piece that will build our AMI. It is made up of a few pieces:

1. The [main.json](https://github.com/mozilla/nubis-dpaste/tree/master/nubis/packer/main.json) file which contains:
    * A *builders* statement describing where to build the AMI
    * One or more *provisioners* statements for calling puppet standalone and any bootstrapping commands for the application
2. A [variables.json](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/packer/variables.json-dist) file which contains things like:
    * AWS credentials
    * base AMI ID

To run packer, from the repository root you first need to create your *variables.json* file, for which there is a template provided (variables.json-dist). After which you simply call packer like so:
```bash
packer build -var-file=nubis/packer/variables.json -var release=0 -var build=1 nubis/packer/main.json
```
This takes around *11m 18.488s* to complete.


### Terraform
The next step is to take the shiny new AMI that Packer built and deploy it. This is where [Terraform](https://www.terraform.io/) comes into play. Terraform is our infrastructure deployment framework, but not to worry it is really not as complicated as its name implies. It consists of a few files:

1. [inputs.tf](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/inputs.tf) simply lists the variables you might need to provide
2. [main.tf](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/main.tf) is where the real heavy lifting takes place. This is where you describe your infrastructure. Thisgs like EC2 instances, security groups, ELBs and so on.
3. [outputs.tf](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/outputs.tf) describes what information from the build we want to make available (via Consul) for later reference.
4. [terraform.tfvars](https://github.com/mozilla/nubis-dpaste/blob/master/nubis/terraform/terraform.tfvars-dist) is where you will set your AWS credentials and such.

To run terraform, from the repository root you first need to create your *terraform.tfvars* file, for which there is a template provided (terraform.tfvars-dist). After which you simply call terraform like so:

To see what resources will be created, destroyed, or refreshed:
```bash
terraform plan -var-file=nubis/terraform/terraform.tfvars nubis/terraform/
```
To apply the plan (do the work)
```bash
{'fraction_non_alphanumeric': 0.08431245965138799, 'fraction_numerical': 0.004131697869593286, 'mean_word_length': 4.5249643366619114, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 23, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3416057', 'n_tokens_mistral': 2224, 'n_tokens_neox': 2186, 'n_words': 970}