<commit_before><!---
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

# Working with IAM Assumed Roles

<!-- MACRO{toc|fromDepth=0|toDepth=2} -->

AWS ["IAM Assumed Roles"](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html)
allows applications to change the AWS role with which to authenticate with AWS services.
The assumed roles can have different rights from the main user login.

The S3A connector supports assumed roles for authentication with AWS.
A full set of login credentials must be provided, which will be used
to obtain the assumed role and refresh it regularly.
By using per-filesystem configuration, it is possible to use different
assumed roles for different buckets.

*IAM Assumed Roles are unlikely to be supported by third-party systems
supporting the S3 APIs.*

## <a name="using_assumed_roles"></a> Using IAM Assumed Roles

### Before You Begin

This document assumes you know about IAM Assumed roles, what they
are, how to configure their policies, etc.

* You need a role to assume, and know its "ARN".
* You need a pair of long-lived IAM User credentials, not the root account set.
* Have the AWS CLI installed, and test that it works there.
* Give the role access to S3.
* For working with data encrypted with SSE-KMS, the role must
have access to the appropriate KMS keys.

Trying to learn how IAM Assumed Roles work by debugging stack traces from
the S3A client is "suboptimal".

### <a name="how_it_works"></a> How the S3A connector supports IAM Assumed Roles.


The S3A connector support IAM Assumed Roles in two ways:

1. Using the full credentials on the client to request credentials for a specific
    role -credentials which are then used for all the store operations.
    This can be used to verify that a specific role has the access permissions
    you need, or to "su" into a role which has permissions that's the full
    accounts does not directly qualify for -such as access to a KMS key.
2. Using the full credentials to request role credentials which are then
    propagated into a launched application as delegation tokens.
    This extends the previous use as it allows the jobs to be submitted to a
    shared cluster with the permissions of the requested role, rather than
    those of the VMs/Containers of the deployed cluster.

For Delegation Token integration, see (Delegation Tokens)[delegation_tokens.html]

To for Assumed Role authentication, the client must be configured to use the
*Assumed Role Credential Provider*, `org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider`,
in the configuration option `fs.s3a.aws.credentials.provider`.

This AWS Credential provider will read in the `fs.s3a.assumed.role` options needed to connect to the
Security Token Service [Assumed Role API](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html),
first authenticating with the full credentials, then assuming the specific role
specified. It will then refresh this login at the configured rate of
`fs.s3a.assumed.role.session.duration`

To authenticate with the  [AWS STS service](https://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html)
both for the initial credential retrieval
and for background refreshes, a different credential provider must be
created, one which uses long-lived credentials (secret keys, environment variables).
Short lived credentials (e.g other session tokens, EC2 instance credentials) cannot be used.

A list of providers can be set in `s.s3a.assumed.role.credentials.provider`;
if unset the standard `BasicAWSCredentialsProvider` credential provider is used,
which uses `fs.s3a.access.key` and `fs.s3a.secret.key`.

Note: although you can list other AWS credential providers in  to the<commit_msg>HADOOP-18114. Documentation correction in assumed_roles.md  (#3949)<commit_after><!---
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

# Working with IAM Assumed Roles

<!-- MACRO{toc|fromDepth=0|toDepth=2} -->

AWS ["IAM Assumed Roles"](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html)
allows applications to change the AWS role with which to authenticate with AWS services.
The assumed roles can have different rights from the main user login.

The S3A connector supports assumed roles for authentication with AWS.
A full set of login credentials must be provided, which will be used
to obtain the assumed role and refresh it regularly.
By using per-filesystem configuration, it is possible to use different
assumed roles for different buckets.

*IAM Assumed Roles are unlikely to be supported by third-party systems
supporting the S3 APIs.*

## <a name="using_assumed_roles"></a> Using IAM Assumed Roles

### Before You Begin

This document assumes you know about IAM Assumed roles, what they
are, how to configure their policies, etc.

* You need a role to assume, and know its "ARN".
* You need a pair of long-lived IAM User credentials, not the root account set.
* Have the AWS CLI installed, and test that it works there.
* Give the role access to S3.
* For working with data encrypted with SSE-KMS, the role must
have access to the appropriate KMS keys.

Trying to learn how IAM Assumed Roles work by debugging stack traces from
the S3A client is "suboptimal".

### <a name="how_it_works"></a> How the S3A connector supports IAM Assumed Roles.


The S3A connector support IAM Assumed Roles in two ways:

1. Using the full credentials on the client to request credentials for a specific
    role -credentials which are then used for all the store operations.
    This can be used to verify that a specific role has the access permissions
    you need, or to "su" into a role which has permissions that's the full
    accounts does not directly qualify for -such as access to a KMS key.
2. Using the full credentials to request role credentials which are then
    propagated into a launched application as delegation tokens.
    This extends the previous use as it allows the jobs to be submitted to a
    shared cluster with the permissions of the requested role, rather than
    those of the VMs/Containers of the deployed cluster.

For Delegation Token integration, see (Delegation Tokens)[delegation_tokens.html]

To for Assumed Role authentication, the client must be configured to use the
*Assumed Role Credential Provider*, `org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider`,
in the configuration option `fs.s3a.aws.credentials.provider`.

This AWS Credential provider will read in the `fs.s3a.assumed.role` options needed to connect to the
Security Token Service [Assumed Role API](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html),
first authenticating with the full credentials, then assuming the specific role
specified. It will then refresh this login at the configured rate of
`fs.s3a.assumed.role.session.duration`

To authenticate with the  [AWS STS service](https://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html)
both for the initial credential retrieval
and for background refreshes, a different credential provider must be
created, one which uses long-lived credentials (secret keys, environment variables).
Short lived credentials (e.g other session tokens, EC2 instance credentials) cannot be used.

A list of providers can be set in `fs.s3a.assumed.role.credentials.provider`;
if unset the standard `BasicAWSCredentialsProvider` credential provider is used,
which uses `fs.s3a.access.key` and `fs.s3a.secret.key`.

Note: although you can list other AWS credential providers in  to the
{'fraction_non_alphanumeric': 0.05925752430026935, 'fraction_numerical': 0.006206815786391849, 'mean_word_length': 4.41191381495564, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4942464', 'n_tokens_mistral': 2289, 'n_tokens_neox': 2223, 'n_words': 1166}