<commit_before>Ansible role for Varda
======================


Requirements
------------

- Debian 8 (Jessie) with a configured MTA
- Ansible version 1.9.2


Description
-----------

The deployment uses the following tools:

- [nginx](http://nginx.org/) as reverse proxy for Gunicorn and for serving
  static files.
- [systemd](http://freedesktop.org/wiki/Software/systemd/) for process
  control.
- [Gunicorn](http://gunicorn.org/) as WSGI HTTP server.
- [virtualenv](http://virtualenv.readthedocs.org/) for isolation of the Varda
  package and its Python dependencies.
- [PostgreSQL](http://www.postgresql.org/) for the database.
- [Redis](http://redis.io/) for message broker and task result backend.

Two applications are served by nginx:

- Varda RESTful HTTP API: `https://<servername>/api`
- Aulë web interface: `https://<servername>/`


### Files and directories

Logging is done at several levels in this stack:

- Varda API Gunicorn log: `/opt/varda/versions/*/log/api.log`
- Varda Celery log: `/opt/varda/versions/*/log/celery.log`
- nginx access and error log: `/var/log/nginx/`
- redis server logs: `/var/log/redis/`
- PostgreSQL server logs: `/var/log/postgresql/`

Tool configurations can be found here (but you should never manually touch
them):

- Varda configuration: `/opt/varda/versions/*/conf/settings.py`
- Aulë configuration: `/opt/varda/src/aule/web/scripts/config.coffee`
- Varda API Gunicorn configuration: `/opt/varda/versions/*/conf/api.conf.py`
- Varda Celery configuration: `/opt/varda/versions/*/conf/celery.conf`
- nginx configuration: `/etc/nginx/sites-available/varda`

All Varda processes run as user `varda`, which is also the owner of everything
under `/opt/varda`. Some other Varda related locations are:

- Varda data: `/opt/varda/data/`
- Varda reference genome: `/opt/varda/genome/`
- Varda Git clone: `/opt/varda/src/varda/`
- Aulë Git clone: `/opt/varda/src/aule/`
- Varda versions: `/opt/varda/versions/`


### Zero-downtime depoloyments

In order to obtain zero-downtime deployments, we fix and isolate deployment
versions. A version is identified by its Git commit hash and contains a Python<commit_msg>Fix typo<commit_after>Ansible role for Varda
======================


Requirements
------------

- Debian 8 (Jessie) with a configured MTA
- Ansible version 1.9.2


Description
-----------

The deployment uses the following tools:

- [nginx](http://nginx.org/) as reverse proxy for Gunicorn and for serving
  static files.
- [systemd](http://freedesktop.org/wiki/Software/systemd/) for process
  control.
- [Gunicorn](http://gunicorn.org/) as WSGI HTTP server.
- [virtualenv](http://virtualenv.readthedocs.org/) for isolation of the Varda
  package and its Python dependencies.
- [PostgreSQL](http://www.postgresql.org/) for the database.
- [Redis](http://redis.io/) for message broker and task result backend.

Two applications are served by nginx:

- Varda RESTful HTTP API: `https://<servername>/api`
- Aulë web interface: `https://<servername>/`


### Files and directories

Logging is done at several levels in this stack:

- Varda API Gunicorn log: `/opt/varda/versions/*/log/api.log`
- Varda Celery log: `/opt/varda/versions/*/log/celery.log`
- nginx access and error log: `/var/log/nginx/`
- redis server logs: `/var/log/redis/`
- PostgreSQL server logs: `/var/log/postgresql/`

Tool configurations can be found here (but you should never manually touch
them):

- Varda configuration: `/opt/varda/versions/*/conf/settings.py`
- Aulë configuration: `/opt/varda/src/aule/web/scripts/config.coffee`
- Varda API Gunicorn configuration: `/opt/varda/versions/*/conf/api.conf.py`
- Varda Celery configuration: `/opt/varda/versions/*/conf/celery.conf`
- nginx configuration: `/etc/nginx/sites-available/varda`

All Varda processes run as user `varda`, which is also the owner of everything
under `/opt/varda`. Some other Varda related locations are:

- Varda data: `/opt/varda/data/`
- Varda reference genome: `/opt/varda/genome/`
- Varda Git clone: `/opt/varda/src/varda/`
- Aulë Git clone: `/opt/varda/src/aule/`
- Varda versions: `/opt/varda/versions/`


### Zero-downtime deployments

In order to obtain zero-downtime deployments, we fix and isolate deployment
versions. A version is identified by its Git commit hash and contains a Python
{'fraction_non_alphanumeric': 0.14818266542404473, 'fraction_numerical': 0.001863932898415657, 'mean_word_length': 3.7806236080178173, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '582284', 'n_tokens_mistral': 1485, 'n_tokens_neox': 1384, 'n_words': 426}