<commit_before>#S3 Transfer Manager Demo

###Set up Cognito
First we will need to set up Cognito, which is the recommended way of 
providing authentication. Do this through the AWS Console.

###Set up permissions
Now that you've set up Cognito, we need to set up the IAM permissions.

1. Go to https://console.aws.amazon.com/iam/home
2. Select Roles
3. Press the "Create New Role" button
4. Choose a name for your role, and then press "Continue"
5. Under "AWS Service Roles", choose "Amazon EC2". (As of this writing there is
no template for using Cognito, so we can actually just choose any that let us 
use a custom policy)
6. Select "Custom Policy"
7. Fill in a name of your choosing for "Policy Name", and then enter this into
the "Policy Document" (Replacing YOUR_BUCKET with your unique bucket's name)
```
<pre>
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::<YOUR_BUCKET>/${cognito-identity.amazonaws.com:sub}/*"
    },{
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<YOUR_BUCKET>",
      "Condition": 
      {"StringLike": 
        {"s3:prefix":"${cognito-identity.amazonaws.com:sub}/*"}
      }
    }
  ]
}
</pre>
```
8. Select "Create Role"
9. Finally, select the role you just made and go to the "Trust Relationships"
tab
10. Click "Edit Trust Relationship"
11. Replace the "Policy Document" with the following
```
<pre>
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Federated": "cognito-identity.amazonaws.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity"
    }
  ]
}
</pre>
```
12. Select "Update Policy" to apply the changes
13. Take note of the "Role ARN" under the "Summary" tab; we'll need this later
<commit_msg>Markdown doesn't use <pre> tags<commit_after>#S3 Transfer Manager Demo

###Set up Cognito
First we will need to set up Cognito, which is the recommended way of 
providing authentication. Do this through the AWS Console.

###Set up permissions
Now that you've set up Cognito, we need to set up the IAM permissions.

1. Go to https://console.aws.amazon.com/iam/home
2. Select Roles
3. Press the "Create New Role" button
4. Choose a name for your role, and then press "Continue"
5. Under "AWS Service Roles", choose "Amazon EC2". (As of this writing there is
no template for using Cognito, so we can actually just choose any that let us 
use a custom policy)
6. Select "Custom Policy"
7. Fill in a name of your choosing for "Policy Name", and then enter this into
the "Policy Document" (Replacing YOUR_BUCKET with your unique bucket's name)
```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::<YOUR_BUCKET>/${cognito-identity.amazonaws.com:sub}/*"
    },{
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<YOUR_BUCKET>",
      "Condition": 
      {"StringLike": 
        {"s3:prefix":"${cognito-identity.amazonaws.com:sub}/*"}
      }
    }
  ]
}
```
8. Select "Create Role"
9. Finally, select the role you just made and go to the "Trust Relationships"
tab
10. Click "Edit Trust Relationship"
11. Replace the "Policy Document" with the following
```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Federated": "cognito-identity.amazonaws.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity"
    }
  ]
}
```
12. Select "Update Policy" to apply the changes
13. Take note of the "Role ARN" under the "Summary" tab; we'll need this later

{'fraction_non_alphanumeric': 0.13755341880341881, 'fraction_numerical': 0.0219017094017094, 'mean_word_length': 3.411071849234393, 'pattern_counts': {'":': 36, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2106031', 'n_tokens_mistral': 1274, 'n_tokens_neox': 1212, 'n_words': 446}