<commit_before># GCS Resource [![Build Status](https://travis-ci.org/frodenas/gcs-resource.png)](https://travis-ci.org/frodenas/gcs-resource)

Versions objects in a [Google Cloud Storage][gcs] (GCS) bucket, by pattern-matching filenames to identify version numbers.

This resource is based on the official [S3 resource][s3-resource].

## Source Configuration

* `bucket` (*required*): the name of the bucket.

* `json_key` (*required*): the contents of your GCS Account JSON Key file to use when accessing the bucket. Example:
  ```
  json_key: |
    {
      "private_key_id": "...",
      "private_key": "...",
      "client_email": "...",
      "client_id": "...",
      "type": "service_account"
    }
  ```

### File Names

One of the following two options must be specified:

* `regexp`: the pattern to match filenames against within GCS. The first grouped match is used to extract the version, or if a group is explicitly named `version`, that group is used.

  The version extracted from this pattern is used to version the resource. Semantic versions, or just numbers, are supported. Accordingly, full regular expressions are supported, to specify the capture groups.

* `versioned_file`: if you [enable versioning][gsc-versioning] for your GCS bucket then you can keep the file name the same and upload new versions of your file without resorting to version numbers. This property is the path to the file in your GCS bucket.

## Behavior

### `check`: Extract versions from the bucket.

Objects will be found via the pattern configured by `regexp` or `versioned_file`. The versions will be used to order them (using [semver][semver]). Each object's filename is the resulting version.

### `in`: Fetch an object from the bucket.

Places the following files in the destination:

* `(filename)`: the file fetched from the bucket.

* `url`: a file containing the URL of the object.

* `version`: the version identified in the file name (only if using `regexp`).

* `generation`: the object's generation (only if using `versioned_file`).

#### Parameters

*None*

### `out`: Upload an object to the bucket.

Given a file specified by `file`, upload it to the GCS bucket. If `regexp` is
specified, the new file will be uploaded to the directory that the regex<commit_msg>Update README.md to reflect "params.unpack" functionality<commit_after># GCS Resource [![Build Status](https://travis-ci.org/frodenas/gcs-resource.png)](https://travis-ci.org/frodenas/gcs-resource)

Versions objects in a [Google Cloud Storage][gcs] (GCS) bucket, by pattern-matching filenames to identify version numbers.

This resource is based on the official [S3 resource][s3-resource].

## Source Configuration

* `bucket` (*required*): the name of the bucket.

* `json_key` (*required*): the contents of your GCS Account JSON Key file to use when accessing the bucket. Example:
  ```
  json_key: |
    {
      "private_key_id": "...",
      "private_key": "...",
      "client_email": "...",
      "client_id": "...",
      "type": "service_account"
    }
  ```

### File Names

One of the following two options must be specified:

* `regexp`: the pattern to match filenames against within GCS. The first grouped match is used to extract the version, or if a group is explicitly named `version`, that group is used.

  The version extracted from this pattern is used to version the resource. Semantic versions, or just numbers, are supported. Accordingly, full regular expressions are supported, to specify the capture groups.

* `versioned_file`: if you [enable versioning][gsc-versioning] for your GCS bucket then you can keep the file name the same and upload new versions of your file without resorting to version numbers. This property is the path to the file in your GCS bucket.

## Behavior

### `check`: Extract versions from the bucket.

Objects will be found via the pattern configured by `regexp` or `versioned_file`. The versions will be used to order them (using [semver][semver]). Each object's filename is the resulting version.

### `in`: Fetch an object from the bucket.

Places the following files in the destination:

* `(filename)`: the file fetched from the bucket.

* `url`: a file containing the URL of the object.

* `version`: the version identified in the file name (only if using `regexp`).

* `generation`: the object's generation (only if using `versioned_file`).

#### Parameters

* `unpack`: *Optional.* If true and the file is an archive (tar, gzipped tar, other gzipped file, or zip), unpack the file. Gzipped tarballs will be both ungzipped and untarred.

### `out`: Upload an object to the bucket.

Given a file specified by `file`, upload it to the GCS bucket. If `regexp` is
specified, the new file will be uploaded to the directory that the regex
{'fraction_non_alphanumeric': 0.1005668696199874, 'fraction_numerical': 0.000839806844425782, 'mean_word_length': 4.122580645161291, 'pattern_counts': {'":': 10, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1805226', 'n_tokens_mistral': 1359, 'n_tokens_neox': 1309, 'n_words': 634}