<commit_before>WAL-E Disaster Recovery
=======================

WAL-E is a program designed perform continuous archiving of PostgreSQL
WAL files and manage the use of pg_start_backup and pg_stop_backup.
It has four critical operators:

* backup-fetch
* backup-push
* wal-fetch
* wal-push

Of these, the "push" operators send things to S3, and "fetch"
operators get things from S3.  "wal" operators send/get write ahead
log, and "backup" send/get a hot backup of the base database that WAL
segments can be applied to.

All of these operators work in a context of three important
environment-variable based settings:

* AWS_ACCESS_KEY_ID
* AWS_SECRET_ACCESS_KEY
* WALE_S3_PREFIX

With the exception of AWS_SECRET_ACCESS_KEY, all of these can be
specified as arguments as well.  The AWS_* variables are the standard
access-control keying system provided by Amazon.

The WALE_S3_PREFIX can be thought of a context whereby this program
operates on a single database cluster at a time.  Generally, for any
one database the WALE_S3_PREFIX will be the same between all four
operators.  This context-driven approach attempts to help users avoid
errors such as one database overwriting the WAL segments of another,
as long as the WALE_S3_PREFIX is set uniquely for each database.

Examples
--------

Pushing a base backup to S3::

  $ AWS_SECRET_KEY=... python wal_e.py			\
    -k AWS_ACCESS_KEY_ID				\
    --s3-prefix=s3://some-bucket/directory/or/whatever	\
    backup_push /var/lib/my/database

Compression and Temporary Files
-------------------------------

All assets pushed to S3 are run through the program "lzop" which
compresses the object using the very fast lzo compression algorithm.
It takes roughly 2 CPU seconds to compress a gigabyte, which when
sending things to S3 at about 25MB/s occupies about 5% CPU time.<commit_msg>Fix documentation bug, misnamed environment variable<commit_after>WAL-E Disaster Recovery
=======================

WAL-E is a program designed perform continuous archiving of PostgreSQL
WAL files and manage the use of pg_start_backup and pg_stop_backup.
It has four critical operators:

* backup-fetch
* backup-push
* wal-fetch
* wal-push

Of these, the "push" operators send things to S3, and "fetch"
operators get things from S3.  "wal" operators send/get write ahead
log, and "backup" send/get a hot backup of the base database that WAL
segments can be applied to.

All of these operators work in a context of three important
environment-variable based settings:

* AWS_ACCESS_KEY_ID
* AWS_SECRET_ACCESS_KEY
* WALE_S3_PREFIX

With the exception of AWS_SECRET_ACCESS_KEY, all of these can be
specified as arguments as well.  The AWS_* variables are the standard
access-control keying system provided by Amazon.

The WALE_S3_PREFIX can be thought of a context whereby this program
operates on a single database cluster at a time.  Generally, for any
one database the WALE_S3_PREFIX will be the same between all four
operators.  This context-driven approach attempts to help users avoid
errors such as one database overwriting the WAL segments of another,
as long as the WALE_S3_PREFIX is set uniquely for each database.

Examples
--------

Pushing a base backup to S3::

  $ AWS_SECRET_ACCESS_KEY=... python wal_e.py		\
    -k AWS_ACCESS_KEY_ID				\
    --s3-prefix=s3://some-bucket/directory/or/whatever	\
    backup_push /var/lib/my/database

Compression and Temporary Files
-------------------------------

All assets pushed to S3 are run through the program "lzop" which
compresses the object using the very fast lzo compression algorithm.
It takes roughly 2 CPU seconds to compress a gigabyte, which when
sending things to S3 at about 25MB/s occupies about 5% CPU time.
{'fraction_non_alphanumeric': 0.07692307692307693, 'fraction_numerical': 0.008097165991902834, 'mean_word_length': 3.908609271523179, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7672974', 'n_tokens_mistral': 1076, 'n_tokens_neox': 1040, 'n_words': 494}