<commit_before># PgBackRest Installation

## sample ubuntu 12.04 install

1. Starting from a clean install, update the OS:

```
apt-get update
apt-get upgrade (reboot if required)
```

2. Install ssh, git and cpanminus

```
apt-get install ssh
apt-get install git
apt-get install cpanminus
```

3. Install Postgres (instructions from http://www.postgresql.org/download/linux/ubuntu/)

Create the file /etc/apt/sources.list.d/pgdg.list, and add a line for the repository:
```
deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main
```
Then run the following:
```
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update

apt-get install postgresql-9.3
apt-get install postgresql-server-dev-9.3
```

4. Install required Perl modules:
```
cpanm JSON
cpanm Net::OpenSSH
cpanm DBI
cpanm DBD::Pg
cpanm IPC::System::Simple
cpanm Digest::SHA
cpanm IO::Compress::Gzip
cpanm IO::Uncompress::Gunzip
```
5. Install PgBackRest

Backrest can be installed by downloading the most recent release:

https://github.com/pgmasters/backrest/releases

6. To run unit tests:

* Create backrest_dev user
* Setup trusted ssh between test user account and backrest_dev
* Backrest user and test user must be in the same group

## configuration examples

PgBackRest takes some command-line parameters, but depends on a configuration file for most of the settings.  The default location for the configuration file is /etc/pg_backrest.conf.

#### confguring postgres for archiving with backrest

Modify the following settings in postgresql.conf:
```
wal_level = archive
archive_mode = on
archive_command = '/path/to/backrest/bin/pg_backrest.pl --stanza=db archive-push %p'
```

Replace the path with the actual location where PgBackRest was installed.  The stanza parameter should be changed to the actual stanza name you used for your database in pg_backrest.conf.

#### simple single host install

This configuration is appropriate for a small installation where backups are being made locally or to a remote file system that is mounted locally.

`/etc/pg_backrest.conf`:
```
[global:command]
psql=/usr/bin/psql

[global:backup]
path=/var/lib/postgresql/backup

[global:retention]
full-retention=2
differential-retention=2
archive-retention-type=diff
archive-retention=2

[db]
path=/var/lib/postgresql/9.3/main
```

#### simple multiple host install

This configuration is appropriate for a small installation where backups are being made remotely.  Make sure that postgres@db-host has trusted ssh to backrest@backup-host and vice versa.  This configuration assumes that you have pg_backrest_remote.pl and pg_backrest.pl in the same path on both servers.

`/etc/pg_backrest.conf on the db host`:
```
[global:command]
psql=/usr/bin/psql

[global:backup]
host=<EMAIL>
user=postgres
path=/var/lib/postgresql/backup

[db]
path=/var/lib/postgresql/9.3/main
```
`/etc/pg_backrest.conf on the backup host`:
```
[global:command]
psql=/usr/bin/psql

[global:backup]
path=/var/lib/postgresql/backup

[global:retention]
full-retention=2
archive-retention-type=full

[db]
host=<EMAIL>
user=postgres
path=/var/lib/postgresql/9.3/main
```

## running

PgBackRest is intended to be run from a scheduler like cron as there is no built-in scheduler.  Postgres does backup rotation, but it is not concerned with when the backups were created.  So if two full backups are configured in rentention, PgBackRest will keep two full backup no matter whether they occur 2 hours apart or two weeks apart.

There are four basic operations:

1. Backup
```
/path/to/pg_backrest.pl --stanza=db --type=full backup
```
Run a `full` backup on the `db` stanza.  `--type` can also be set to `incr` or `diff` for incremental or differential backups.  However, if now `full` backup exists then a `full` backup will be forced even if `incr`

2. Archive Push
```
/path/to/pg_backrest.pl --stanza=db archive-push %p
```
Accepts an archive file from Postgres and pushes it to the backup.  `%p` is how Postgres specifies the location of the file to be archived.  This command has no other purpose.

3.  Archive Get
```
/path/to/pg_backrest.pl --stanza=db archive-get %f %p
```
Retrieves an archive log from the backup.  This is used in `restore.conf` to restore a backup to that last archive log, do PITR, or as an alternative to streaming for keep a replica up to date.  `%f` is how Postgres specifies the archive log it needs, and `%p` is the location where it should be copied.

3. Backup Expire
```
/path/to/pg_backrest.pl --stanza=db expire
```
Expire (rotate) any backups that exceed the defined retention.  Expiration is run after every backup, so there's no need to run this command on its own unless you have reduced rentention, usually to free up some space.

## structure

PgBackRest stores files in a way that is easy for users to work with directly.  Each backup directory has two files and two subdirectories:

1. `backup.manifest` file

Stores information about all the directories, links, and files in the backup.  The file is plaintext and should be very clear, but documentation of the format is planned in a future release.

2. `version` file

Contains the PgBackRest version that was used to create the backup.

3. `base` directory

Contains the Postgres data directory as defined by the data_directory setting in postgresql.conf

4. `tablespace` directory

Contains each tablespace in a separate subdirectory.  The links in `base/pg_tblspc` are rewritten to this directory.

## restoring

PgBackRest does not currently have a restore command - this is planned for the near future.  However, PgBackRest stores backups in a way that makes restoring very easy.  If `compress=n` it is even possible to start Postgres directly on the backup directory.

In order to restore a backup, simple rsync the files from the base backup directory to your data directory.  If you have used compression, then recursively ungzip the files.  If you have tablespaces, repeat the process for each tablespace in the backup tablespace directory.

It's good to practice restoring backups in advance of needing to do so.

## configuration options

Each section defines important aspects of the backup.  All configuration sections below should be prefixed with `global:` as demonstrated in the configuration samples.

#### command section

The command section defines external commands that are used by PgBackRest.

##### psql key

Defines the full path to psql.  psql is used to call pg_start_backup() and pg_stop_backup().
```
required: y
example: psql=/usr/bin/psql
```
##### remote key

Defines the file path to pg_backrest_remote.pl.

Required only if the path to pg_backrest_remote.pl is different on the local and remote systems.  If not defined, the remote path will be assumed to be the same as the local path.
```
required: n
example: remote=/home/postgres/backrest/bin/pg_backrest_remote.pl
```
#### command-option section

The command-option section allows abitrary options to be passed to any command in the command section.<commit_msg>Fixed typos.<commit_after># PgBackRest Installation

## sample ubuntu 12.04 install

1. Starting from a clean install, update the OS:

```
apt-get update
apt-get upgrade (reboot if required)
```

2. Install ssh, git and cpanminus

```
apt-get install ssh
apt-get install git
apt-get install cpanminus
```

3. Install Postgres (instructions from http://www.postgresql.org/download/linux/ubuntu/)

Create the file /etc/apt/sources.list.d/pgdg.list, and add a line for the repository:
```
deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main
```
Then run the following:
```
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update

apt-get install postgresql-9.3
apt-get install postgresql-server-dev-9.3
```

4. Install required Perl modules:
```
cpanm JSON
cpanm Net::OpenSSH
cpanm DBI
cpanm DBD::Pg
cpanm IPC::System::Simple
cpanm Digest::SHA
cpanm IO::Compress::Gzip
cpanm IO::Uncompress::Gunzip
```
5. Install PgBackRest

Backrest can be installed by downloading the most recent release:

https://github.com/pgmasters/backrest/releases

6. To run unit tests:

* Create backrest_dev user
* Setup trusted ssh between test user account and backrest_dev
* Backrest user and test user must be in the same group

## configuration examples

PgBackRest takes some command-line parameters, but depends on a configuration file for most of the settings.  The default location for the configuration file is /etc/pg_backrest.conf.

#### confguring postgres for archiving with backrest

Modify the following settings in postgresql.conf:
```
wal_level = archive
archive_mode = on
archive_command = '/path/to/backrest/bin/pg_backrest.pl --stanza=db archive-push %p'
```

Replace the path with the actual location where PgBackRest was installed.  The stanza parameter should be changed to the actual stanza name you used for your database in pg_backrest.conf.

#### simple single host install

This configuration is appropriate for a small installation where backups are being made locally or to a remote file system that is mounted locally.

`/etc/pg_backrest.conf`:
```
[global:command]
psql=/usr/bin/psql

[global:backup]
path=/var/lib/postgresql/backup

[global:retention]
full-retention=2
differential-retention=2
archive-retention-type=diff
archive-retention=2

[db]
path=/var/lib/postgresql/9.3/main
```

#### simple multiple host install

This configuration is appropriate for a small installation where backups are being made remotely.  Make sure that postgres@db-host has trusted ssh to backrest@backup-host and vice versa.  This configuration assumes that you have pg_backrest_remote.pl and pg_backrest.pl in the same path on both servers.

`/etc/pg_backrest.conf on the db host`:
```
[global:command]
psql=/usr/bin/psql

[global:backup]
host=<EMAIL>
user=postgres
path=/var/lib/postgresql/backup

[db]
path=/var/lib/postgresql/9.3/main
```
`/etc/pg_backrest.conf on the backup host`:
```
[global:command]
psql=/usr/bin/psql

[global:backup]
path=/var/lib/postgresql/backup

[global:retention]
full-retention=2
archive-retention-type=full

[db]
host=<EMAIL>
user=postgres
path=/var/lib/postgresql/9.3/main
```

## running

PgBackRest is intended to be run from a scheduler like cron as there is no built-in scheduler.  Postgres does backup rotation, but it is not concerned with when the backups were created.  So if two full backups are configured in rentention, PgBackRest will keep two full backup no matter whether they occur 2 hours apart or two weeks apart.

There are four basic operations:

1. Backup
```
/path/to/pg_backrest.pl --stanza=db --type=full backup
```
Run a `full` backup on the `db` stanza.  `--type` can also be set to `incr` or `diff` for incremental or differential backups.  However, if now `full` backup exists then a `full` backup will be forced even if `incr`

2. Archive Push
```
/path/to/pg_backrest.pl --stanza=db archive-push %p
```
Accepts an archive file from Postgres and pushes it to the backup.  `%p` is how Postgres specifies the location of the file to be archived.  This command has no other purpose.

3.  Archive Get
```
/path/to/pg_backrest.pl --stanza=db archive-get %f %p
```
Retrieves an archive log from the backup.  This is used in `restore.conf` to restore a backup to that last archive log, do PITR, or as an alternative to streaming for keep a replica up to date.  `%f` is how Postgres specifies the archive log it needs, and `%p` is the location where it should be copied.

3. Backup Expire
```
/path/to/pg_backrest.pl --stanza=db expire
```
Expire (rotate) any backups that exceed the defined retention.  Expiration is run after every backup, so there's no need to run this command on its own unless you have reduced rentention, usually to free up some space.

## structure

PgBackRest stores files in a way that is easy for users to work with directly.  Each backup directory has two files and two subdirectories:

1. `backup.manifest` file

Stores information about all the directories, links, and files in the backup.  The file is plaintext and should be very clear, but documentation of the format is planned in a future release.

2. `version` file

Contains the PgBackRest version that was used to create the backup.

3. `base` directory

Contains the Postgres data directory as defined by the data_directory setting in postgresql.conf

4. `tablespace` directory

Contains each tablespace in a separate subdirectory.  The links in `base/pg_tblspc` are rewritten to this directory.

## restoring

PgBackRest does not currently have a restore command - this is planned for the near future.  However, PgBackRest stores backups in a way that makes restoring very easy.  If `compress=n` it is even possible to start Postgres directly on the backup directory.

In order to restore a backup, simply rsync the files from the base backup directory to your data directory.  If you have used compression, then recursively ungzip the files.  If you have tablespaces, repeat the process for each tablespace in the backup tablespace directory.

It's best to practice restoring backups in advance of needing to do so.

## configuration options

Each section defines important aspects of the backup.  All configuration sections below should be prefixed with `global:` as demonstrated in the configuration samples.

#### command section

The command section defines external commands that are used by PgBackRest.

##### psql key

Defines the full path to psql.  psql is used to call pg_start_backup() and pg_stop_backup().
```
required: y
example: psql=/usr/bin/psql
```
##### remote key

Defines the file path to pg_backrest_remote.pl.

Required only if the path to pg_backrest_remote.pl is different on the local and remote systems.  If not defined, the remote path will be assumed to be the same as the local path.
```
required: n
example: remote=/home/postgres/backrest/bin/pg_backrest_remote.pl
```
#### command-option section

The command-option section allows abitrary options to be passed to any command in the command section.
{'fraction_non_alphanumeric': 0.08124244793517663, 'fraction_numerical': 0.004975478001279408, 'mean_word_length': 4.430335777692011, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6011733', 'n_tokens_mistral': 4298, 'n_tokens_neox': 4111, 'n_words': 1832}