<commit_before># Infrastructure
Set up infrastructure for Weenhanceit projects.

## Architecture
One server with multiple static sites and/or Rails applications,
and one database instance on another server.

On the application server,
Nginx proxies the requests through to the appropriate Rails application,
or serves up the pages of the appropriate static web site.
There is an instance of Puma for each of the Rails applications.

Each Rails application has its database configuration
to talk to a database on the database server.

For applications that need Redis,
we run a separate Redis instance for each application.
The Redis instance listens on a domain socket
at `/tmp/redis.domain_name`.

## Prerequisites
This document assumes you've already set up a server,
and a Postgres instance.

## Common First Steps
These steps are necessary whether you want to deploy
a static web site
or a Rails application.

1. Log in to the server via SSH
```
    ssh -i ~/.ssh/weita.pem ubuntu@URL-of-server
```
2. Get the set-up scripts:
```
    wget https://github.com/weenhanceit/infrastructure/archive/master.zip
```
3. Unzip it:
```
    unzip master.zip
```
4. Change to the scripts directory to save some typing:
```
    cd infrastructure-master/basic-app-server
```

## One-Time Server Set-Up
This step is only necessary after you first set up the server.
It installs additional software needed on the server.
```
./build.sh
```

Also install the gem:
```
sudo gem install shared-infrastructure --no-document
```

### Creating Users
Here's how to create named users with sudo privileges on the server.
These instructions work if your desktop is running Ubuntu.

1. Create a key pair if you don't already have one. Accept the defaults,
and don't enter a pass phrase:
```
    mkdir ~/.ssh
    chmod 700 ~/.ssh  
    ssh-keygen -t rsa
```
  This leaves a key pair in `~/.ssh/id_rsa` (the private key)
  and `~/.ssh/id_rsa.pub` (the public key).

2. Obtain a copy of the user creation script:
```
    wget https://github.com/weenhanceit/infrastructure/raw/master/basic-app-server/create-user.sh
```
3. Run the user creation script:
```
    ./create-user -k existing-user-key-file username ~/.ssh/id_rsa.pub ubuntu@ec2-server-address
```
  It's important to understand which key file is which.
  `existing-user-key-file` is the private key that you obtained
  from Amazon when you created the instance.
  `~/.ssh/id_rsa.pub` is the *public* key file
  from the key pair you just generated.

## Creating a Static Web Site
This sets up an Nginx server block for a given domain name.
In all the examples that follow,
replace `domain-name` with your domain name.
```
sudo bundle exec create-server-block domain-name
```
The root directory of the static web site files is `/var/www/domain-name/html`.

Now you can deploy the static web site. [TODO: How to deploy.]

Once deployed, remember to reload the Nginx configuration:
```
sudo nginx -s reload
```

## Creating a Rails Application
This sets up:

* An Nginx server block for a given domain name. The server block proxies via a domain socket to Puma
* A `systemd` service file that runs an instance of Puma
receiving requests on the same domain socket.

### Set the Rails Environment Variables
The Rails instance probably needs secrets. Typically they can be in environment variables (e.g. a Heroku deployment), or in a `secrets.yml` that's put on the server manaully (e.g. Capistrano deployments). For the environment option, `create-rails-app`  will use environment variables if they're set, or prompt you for them if not.

For the following, you get the "secret-key-base" by doing `rails secret`.
[TODO: `rails secret` assumes you have an application set up, which you might not have at this point.]
The "database-username" and "database-password" can be whatever you choose them to be.
```
export SECRET_KEY_BASE=secret-key-base
export DATABASE_USERNAME=database-username
export DATABASE_PASSWORD=<PASSWORD>
export EMAIL_PASSWORD=<PASSWORD>-password
```

### Create the Rails Application
If the application does *not* use `send_file` to ask Nginx to send private files:
```
sudo -E create-rails-app domain-name
```
If the application uses uses `send_file` to ask Nginx to send private files, add the `-a` flag:
```
sudo -E create-rails-app -a location domain-name
```
Where `location` is the `Rails.root` directory. Note that if you have symlinks in the path to `Rails.root`, `Rails.root` contains the real directories. Therefore, in a typical Capistrano deployment, the location is `releases`, not `current`. Nginx won't serve unauthorized files, because it's up to the application to return on file locations that Nginx should serve.

If you're passing the secrets in environment variables, don't forget the `-E` to `sudo`. It causes the environment variables to be passed to the script. It's optional if you just type the variables in.
If you forget to use the `-a` flag,
you can safely re-run this script later with the flag.

The above will tell you how to get a certificate for the site,
but you can't do that yet.
You need to deploy the application the first time,
so the directories get created.

Finally, set up the database:
```
export DATABASE=database
./create-db-user.sh
```
The last step above will ask you for the password for the `root` user in the Postgres database.

The root directory of the Rails application is `/var/www/domain-name/html`.

Now you can deploy the Rails app.
Note that before you deploy,
you have to do `bundle binstubs puma`
in your Rails app, and then commit the `bin` directory to Github. [TODO: How to deploy.]

NOTE: Currently the first deploy will fail,
since the database doesn't exist.
You have to manually do `rails db:setup`
after getting the application code to the server.

Once deployed, remember to reload the Nginx configuration:
```
sudo nginx -s reload
```
The deployment script should start the Puma service.
Check Puma's status with:
```
sudo systemctl status domain-name
```
If it's not running, try to restart it with:<commit_msg>Add note about Capistrano deploys.<commit_after># Infrastructure
Set up infrastructure for Weenhanceit projects.

## Architecture
One server with multiple static sites and/or Rails applications,
and one database instance on another server.

On the application server,
Nginx proxies the requests through to the appropriate Rails application,
or serves up the pages of the appropriate static web site.
There is an instance of Puma for each of the Rails applications.

Each Rails application has its database configuration
to talk to a database on the database server.

For applications that need Redis,
we run a separate Redis instance for each application.
The Redis instance listens on a domain socket
at `/tmp/redis.domain_name`.

## Prerequisites
This document assumes you've already set up a server,
and a Postgres instance.

## Common First Steps
These steps are necessary whether you want to deploy
a static web site
or a Rails application.

1. Log in to the server via SSH
```
    ssh -i ~/.ssh/weita.pem ubuntu@URL-of-server
```
2. Get the set-up scripts:
```
    wget https://github.com/weenhanceit/infrastructure/archive/master.zip
```
3. Unzip it:
```
    unzip master.zip
```
4. Change to the scripts directory to save some typing:
```
    cd infrastructure-master/basic-app-server
```

## One-Time Server Set-Up
This step is only necessary after you first set up the server.
It installs additional software needed on the server.
```
./build.sh
```

Also install the gem:
```
sudo gem install shared-infrastructure --no-document
```

### Creating Users
Here's how to create named users with sudo privileges on the server.
These instructions work if your desktop is running Ubuntu.

1. Create a key pair if you don't already have one. Accept the defaults,
and don't enter a pass phrase:
```
    mkdir ~/.ssh
    chmod 700 ~/.ssh  
    ssh-keygen -t rsa
```
  This leaves a key pair in `~/.ssh/id_rsa` (the private key)
  and `~/.ssh/id_rsa.pub` (the public key).

2. Obtain a copy of the user creation script:
```
    wget https://github.com/weenhanceit/infrastructure/raw/master/basic-app-server/create-user.sh
```
3. Run the user creation script:
```
    ./create-user -k existing-user-key-file username ~/.ssh/id_rsa.pub ubuntu@ec2-server-address
```
  It's important to understand which key file is which.
  `existing-user-key-file` is the private key that you obtained
  from Amazon when you created the instance.
  `~/.ssh/id_rsa.pub` is the *public* key file
  from the key pair you just generated.

## Creating a Static Web Site
This sets up an Nginx server block for a given domain name.
In all the examples that follow,
replace `domain-name` with your domain name.
```
sudo bundle exec create-server-block domain-name
```
The root directory of the static web site files is `/var/www/domain-name/html`.

Now you can deploy the static web site. [TODO: How to deploy.]

Once deployed, remember to reload the Nginx configuration:
```
sudo nginx -s reload
```

## Creating a Rails Application
This sets up:

* An Nginx server block for a given domain name. The server block proxies via a domain socket to Puma
* A `systemd` service file that runs an instance of Puma
receiving requests on the same domain socket.

### Set the Rails Environment Variables
The Rails instance probably needs secrets. Typically they can be in environment variables (e.g. a Heroku deployment), or in a `secrets.yml` that's put on the server manaully (e.g. Capistrano deployments). For the environment option, `create-rails-app`  will use environment variables if they're set, or prompt you for them if not.

For the following, you get the "secret-key-base" by doing `rails secret`.
[TODO: `rails secret` assumes you have an application set up, which you might not have at this point.]
The "database-username" and "database-password" can be whatever you choose them to be.
```
export SECRET_KEY_BASE=secret-key-base
export DATABASE_USERNAME=database-username
export DATABASE_PASSWORD=<PASSWORD>
export EMAIL_PASSWORD=<PASSWORD>
```

### Create the Rails Application
If the application does *not* use `send_file` to ask Nginx to send private files:
```
sudo -E create-rails-app domain-name
```
If the application uses uses `send_file` to ask Nginx to send private files, add the `-a` flag:
```
sudo -E create-rails-app -a location domain-name
```
Where `location` is the `Rails.root` directory. Note that if you have symlinks in the path to `Rails.root`, `Rails.root` contains the real directories. Therefore, in a typical Capistrano deployment, the location is `releases`, not `current`. Nginx won't serve unauthorized files, because it's up to the application to return on file locations that Nginx should serve.

If you're passing the secrets in environment variables, don't forget the `-E` to `sudo`. It causes the environment variables to be passed to the script. It's optional if you just type the variables in.
If you forget to use the `-a` flag,
you can safely re-run this script later with the flag.

The above will tell you how to get a certificate for the site,
but you can't do that yet.
You need to deploy the application the first time,
so the directories get created.

Finally, set up the database:
```
export DATABASE=database
./create-db-user.sh
```
The last step above will ask you for the password for the `root` user in the Postgres database.

The root directory of the Rails application is `/var/www/domain-name/html`.

Now you can deploy the Rails app.
Note that before you deploy,
you have to do `bundle binstubs puma`
in your Rails app, and then commit the `bin` directory to Github. [TODO: How to deploy.]

NOTE: Currently the first deploy will fail,
since the database doesn't exist.
You have to manually do `rails db:setup`
after getting the application code to the server.

(And if you're using Capistrano for the deploy, then you need to manually create the user and database:
```
psql -h <DB host name> -U <master user> -d postgres
create role <user name for application>
  with createdb
  login password '<<PASSWORD>>';
grant rds_superuser to <user name for application>;
\q
psql -h <DB host name> -U <user name for application> -d postgres
<enter password for application user>
create database <database for application>;
\q
```
)

Once deployed, remember to reload the Nginx configuration:
```
sudo nginx -s reload
```
The deployment script should start the Puma service.
Check Puma's status with:
```
sudo systemctl status domain-name
```
If it's not running, try to restart it with:
{'fraction_non_alphanumeric': 0.08084834954552703, 'fraction_numerical': 0.0017541062031573912, 'mean_word_length': 4.130061349693252, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '388758', 'n_tokens_mistral': 3725, 'n_tokens_neox': 3442, 'n_words': 1801}