<commit_before># s3pub

A tiny program for publishing content to [Amazon S3], especially for web hosting.

Given a path to local content on disk, `s3pub` will replicate it in S3, with rsync-like selection for only changed files. The created keys in S3 will resemble those on the local filesystem. If desired, `s3pub` will also trigger an [Amazon CloudFront] invalidation, in order to clear prior caches.

### Example

```
s3pub web-stuff/ mybucket/apath
```

## Credentials

You can provide your Amazon S3 credentials to `s3pub` in two ways:

1. From a file, specified by the `--config-file` option
1. On the command line, via the `--access-key` and `--secret-key` options

In the first case, the contents of the file should look like:

    aws_access_key: YOUR_ACCESS_KEY
    aws_secret_key: YOUR_SECRET_KEY

Credentials specified via the `--access-key` and `--secret-key` options override values read from the credentials file, if any.

You can create a restricted set of credentials for use with `s3pub` via [IAM].

## Installation

1. Download this repository as a zipfile
1. Unpack the archive
1. Create a virtualenv for `s3pub`:

 `python ~/src/virtualenv-1.11.6/virtualenv.py /tmp/v`
1. Install into this virtualenv the dependencies in `requirements/default.txt`:

 `/tmp/v/bin/pip install -r s3pub/requirements/default.txt`

1. Install s3pub:

 `/tmp/v/bin/pip install s3pub/`

1. You can now use `s3pub` from the virtualenv:
  
  `/tmp/v/bin/s3pub --help`
  
In the example commands above, I've used:

1. `/tmp/v` as the path to the virtualenv
1. `~/src/virtualenv-1.11.6/virtualenv.py` as the path to the virtualenv script
1. `s3pub` as the path to the expanded zipfile

## Running tests

[![Build Status](https://travis-ci.org/marmida/s3pub.svg?branch=master)](https://travis-ci.org/marmida/s3pub)

`s3pub` has a fledgling test suite. There are both unit tests and functional tests. To run these:

1. install into your virtualenv the dependencies in `requirements/development.txt`.
1. cd into the s3pub directory.
1. Run `nosetests` for unit tests and `behave` for functional tests. Note that the `behave` test suite requires configuration and write access to S3, as it does actually upload files.

[IAM]: http://aws.amazon.com/iam/
[Amazon S3]: http://aws.amazon.com/s3/
[Amazon CloudFront]: http://aws.amazon.com/cloudfront/

## Alternatives

Since writing this, I've discovered a more feature-complete analog: [s3_website](https://github.com/laurilehmijoki/s3_website).<commit_msg>add link to a similar project<commit_after># s3pub

A tiny program for publishing content to [Amazon S3], especially for web hosting.

Given a path to local content on disk, `s3pub` will replicate it in S3, with rsync-like selection for only changed files. The created keys in S3 will resemble those on the local filesystem. If desired, `s3pub` will also trigger an [Amazon CloudFront] invalidation, in order to clear prior caches.

### Example

```
s3pub web-stuff/ mybucket/apath
```

## Credentials

You can provide your Amazon S3 credentials to `s3pub` in two ways:

1. From a file, specified by the `--config-file` option
1. On the command line, via the `--access-key` and `--secret-key` options

In the first case, the contents of the file should look like:

    aws_access_key: YOUR_ACCESS_KEY
    aws_secret_key: YOUR_SECRET_KEY

Credentials specified via the `--access-key` and `--secret-key` options override values read from the credentials file, if any.

You can create a restricted set of credentials for use with `s3pub` via [IAM].

## Installation

1. Download this repository as a zipfile
1. Unpack the archive
1. Create a virtualenv for `s3pub`:

 `python ~/src/virtualenv-1.11.6/virtualenv.py /tmp/v`
1. Install into this virtualenv the dependencies in `requirements/default.txt`:

 `/tmp/v/bin/pip install -r s3pub/requirements/default.txt`

1. Install s3pub:

 `/tmp/v/bin/pip install s3pub/`

1. You can now use `s3pub` from the virtualenv:
  
  `/tmp/v/bin/s3pub --help`
  
In the example commands above, I've used:

1. `/tmp/v` as the path to the virtualenv
1. `~/src/virtualenv-1.11.6/virtualenv.py` as the path to the virtualenv script
1. `s3pub` as the path to the expanded zipfile

## Running tests

[![Build Status](https://travis-ci.org/marmida/s3pub.svg?branch=master)](https://travis-ci.org/marmida/s3pub)

`s3pub` has a fledgling test suite. There are both unit tests and functional tests. To run these:

1. install into your virtualenv the dependencies in `requirements/development.txt`.
1. cd into the s3pub directory.
1. Run `nosetests` for unit tests and `behave` for functional tests. Note that the `behave` test suite requires configuration and write access to S3, as it does actually upload files.

## Alternatives

Since writing this, I've discovered a more feature-complete analog: [s3_website](https://github.com/laurilehmijoki/s3_website).

...and a less-featureful alternative: [alotofeffort](https://github.com/audreyr/alotofeffort)

[IAM]: http://aws.amazon.com/iam/
[Amazon S3]: http://aws.amazon.com/s3/
[Amazon CloudFront]: http://aws.amazon.com/cloudfront/
{'fraction_non_alphanumeric': 0.10331307586747697, 'fraction_numerical': 0.01881983924720643, 'mean_word_length': 4.0766169154228855, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '391896', 'n_tokens_mistral': 1668, 'n_tokens_neox': 1613, 'n_words': 643}