<commit_before>s3-tools
=======

Collection of multiple s3 tools

s3-copy
=======

s3-copy will copy the content of Amazon S3 buckets from one account to the same or another account. 

```
go get github.com/goamz/goamz/aws
go get github.com/goamz/goamz/s3

go run s3-copy.go s3://ACCESS_KEY:SECRET_KEY@source s3://ACCESS_KEY:SECRET_KEY@destination
```

s3-delete
=======

s3-delete will bulk delete the contents of an Amazon S3 bucket

```
go get github.com/goamz/goamz/aws
go get github.com/goamz/goamz/s3

go run s3-delete.go s3://ACCESS_KEY:SECRET_KEY@source
```

s3-archive
=======

s3-archive will archive files older then 1 month to Amazon S3 bucket with optional move

```
go get github.com/goamz/goamz/aws
go get github.com/goamz/goamz/s3

export AWS_ACCESS_KEY={AWS_ACCESS_KEY}
export AWS_SECRET_KEY={AWS_SECRET_KEY}
export BUCKET={BUCKET}

go run s3-archive --destination=dest --move=false /tmp/
```


If your password contains special characters, you need to url encode them. For example / will be %2F.

TODO
====

* support other regions
* support for paths
* support for wildcards
* add progress
* add statistics (upload speed)
* etc

CONTRIBUTIONS
============

Contributions are welcome.
<commit_msg>added travis<commit_after>s3-tools
=======

Collection of multiple s3 tools

[![Build Status](https://travis-ci.org/dutchcoders/s3-tools.svg?branch=master)](https://travis-ci.org/dutchcoders/s3-tools)

s3-copy
=======

s3-copy will copy the content of Amazon S3 buckets from one account to the same or another account. 

```
go get github.com/goamz/goamz/aws
go get github.com/goamz/goamz/s3

go run s3-copy.go s3://ACCESS_KEY:SECRET_KEY@source s3://ACCESS_KEY:SECRET_KEY@destination
```

s3-delete
=======

s3-delete will bulk delete the contents of an Amazon S3 bucket

```
go get github.com/goamz/goamz/aws
go get github.com/goamz/goamz/s3

go run s3-delete.go s3://ACCESS_KEY:SECRET_KEY@source
```

s3-archive
=======

s3-archive will archive files older then 1 month to Amazon S3 bucket with optional move

```
go get github.com/goamz/goamz/aws
go get github.com/goamz/goamz/s3

export AWS_ACCESS_KEY={AWS_ACCESS_KEY}
export AWS_SECRET_KEY={AWS_SECRET_KEY}
export BUCKET={BUCKET}

go run s3-archive --destination=dest --move=false /tmp/
```


If your password contains special characters, you need to url encode them. For example / will be %2F.

TODO
====

* support other regions
* support for paths
* support for wildcards
* add progress
* add statistics (upload speed)
* etc

CONTRIBUTIONS
============

Contributions are welcome.

{'fraction_non_alphanumeric': 0.12685982772122162, 'fraction_numerical': 0.018010963194988253, 'mean_word_length': 4.482832618025751, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6897697', 'n_tokens_mistral': 951, 'n_tokens_neox': 909, 'n_words': 263}