<commit_before>## Storage Configuration

For image and volume data uploads, neuron-catalog depends on an the
[Amazon Simple Storage Service](http://aws.amazon.com/s3/) or a
compatible provider. Under the hood, this support is provided by
[Meteor
Slingshot](https://github.com/CulturalMe/meteor-slingshot).

### Configuration of Amazon Web Services

You need to setup and configure this to run your own instance of
neuron-catalog.

- Create an AWS user and login to the [AWS Console](https://console.aws.amazon.com/).

- Create an S3 Bucket.

- In the bucket permissions, add the following CORS configuration:

```xml
<pre>
    <code>
       <?xml version="1.0" encoding="UTF-8"?>
       <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
           <CORSRule>
               <AllowedOrigin>*</AllowedOrigin>
               <AllowedMethod>PUT</AllowedMethod>
               <AllowedMethod>POST</AllowedMethod>
               <AllowedMethod>GET</AllowedMethod>
               <AllowedMethod>HEAD</AllowedMethod>
               <MaxAgeSeconds>3000</MaxAgeSeconds>
               <AllowedHeader>*</AllowedHeader>
           </CORSRule>
       </CORSConfiguration>
    </code>
</pre>
```

- Enable static website hosting for this S3 bucket by selecting
`Enable website hosting` in the AWS Console. Also set "Index Document"
to `index.html`.

- Set the bucket policy to (substitute the name of your bucket for
`your-bucket-name`):

<pre>
   <code>
   {
   	"Version": "2008-10-17",
   	"Statement": [
   		{
   			"Sid": "AllowPublicRead",
   			"Effect": "Allow",
   			"Principal": {
   				"AWS": "*"
   			},
   			"Action": "s3:GetObject",
   			"Resource": "arn:aws:s3:::your-bucket-name/*"
   		}
   	]
   }
   </code>
</pre>

- In the Identity & Access Management (IAM) configuration, create a
user and group for performing the uploads. Note the Access Key and the
Secret Key - you will need to enter these.

- Still in IAM, set the group policy to the following (again,
substitute the name of your bucket for `your-bucket-name`):

<pre>
    <code>
       {
         "Version": "2012-10-17",
         "Statement": [
           {
             "Sid": "Stmt1410709913000",
             "Effect": "Allow",
             "Action": [
               "s3:*"
             ],
             "Resource": [
               "arn:aws:s3:::your-bucket-name"
             ]
           },
           {
             "Sid": "Stmt1410710014000",
             "Effect": "Allow",
             "Action": [
               "s3:*"
             ],
             "Resource": [
               "arn:aws:s3:::your-bucket-name/*"
             ]
           }
         ]
       }
    </code>
</pre>

- Now, you need to tell neuron-catalog what your IAM Access Key and
Secret Key created above. These go in a JSON file like the prototype
in `server/server-config.json.example`. Change the relevant variables
and save it as something like `server/server-config.json`. You can
also enter these values into the top of the `Vagrantfile` and
re-create your vagrant machine.<commit_msg>further documentation improvements<commit_after>## Storage Configuration

For image and volume data uploads, neuron-catalog depends on an the
[Amazon Simple Storage Service](http://aws.amazon.com/s3/) or a
compatible provider. Under the hood, this support is provided by
[Meteor
Slingshot](https://github.com/CulturalMe/meteor-slingshot).

### Configuration of Amazon Web Services

You need to setup and configure this to run your own instance of
neuron-catalog.

- Create an AWS user and login to the [AWS Console](https://console.aws.amazon.com/).

- Create an S3 Bucket.

- In the bucket permissions, add the following CORS configuration:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedMethod>POST</AllowedMethod>
        <AllowedMethod>GET</AllowedMethod>
        <AllowedMethod>HEAD</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>
```

- Enable static website hosting for this S3 bucket by selecting
`Enable website hosting` in the AWS Console. Also set "Index Document"
to `index.html`.

- Set the bucket policy to (substitute the name of your bucket for
`your-bucket-name`):

<pre>
   <code>
{
	"Version": "2008-10-17",
	"Statement": [
		{
			"Sid": "AllowPublicRead",
			"Effect": "Allow",
			"Principal": {
				"AWS": "*"
			},
			"Action": "s3:GetObject",
			"Resource": "arn:aws:s3:::your-bucket-name/*"
		}
	]
}
   </code>
</pre>

- In the Identity & Access Management (IAM) configuration, create a
user and group for performing the uploads. Note the Access Key and the
Secret Key - you will need to enter these.

- Still in IAM, set the group policy to the following (again,
substitute the name of your bucket for `your-bucket-name`):

<pre>
    <code>
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1410709913000",
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket-name"
      ]
    },
    {
      "Sid": "Stmt1410710014000",
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket-name/*"
      ]
    }
  ]
}
    </code>
</pre>

- Now, you need to tell neuron-catalog what your IAM Access Key and
Secret Key created above. Please see the [settings](settings.md)
documentation.
{'fraction_non_alphanumeric': 0.13095664636331064, 'fraction_numerical': 0.024005732712289504, 'mean_word_length': 2.56741214057508, 'pattern_counts': {'":': 36, '<': 61, '<?xml version=': 2, '>': 61, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 6}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6768154', 'n_tokens_mistral': 1824, 'n_tokens_neox': 1728, 'n_words': 511}