<commit_before>goawsroles
==========

`goawsroles` provides an interface and sample implementations for safe handling of AWS
credentials. While typically used with the `godynamo` package (https://github.com/smugmug/godynamo),
it is useful on its own and should be easy to integrate into any package requiring a
consistent interface for AWS credentials.

Typically, AWS credentials consist of a AccessKey, Secret and optionally a Token if
temporary credentials are used. We want to be able to safely update these and access
them in programs. It is useful to have a variety of mechanisms for sourcing these credentials -
hardcoded, environment variables, or local files. `goawsroles` seeks to provide one interface
for these and other scenarios, with the goal being safe access to read and update credential data.

At SmugMug, our typical use involves local files in `/etc` for each of the AccessKey, Secret
and Token. These are updated by another process every thirty minutes. Local files provide
for ease-of-use with some decent security via user and group permissions that are more
effective than environment-variable based solutions.  The `RolesFiles`
implementation included here provides a solution for this scenario; it includes a
`RolesWatch` method which, when launched via a goroutine, will watch the local
permissions files for updates. When all three files are updated within a sane time
interval, the internal representation of the permission strings for the AccessKey,
Secret and Token will be updated. Using this mechanism along with another process
to keep our local files updated and valid, we can be sure that any time we call
the `Get` method on the `RolesFiles` instance, we are getting fresh credentials.

### RolesFiles Example

This example is contained within the unit test for the `RolesFiles` instance, with
some extra comments.

        func TestRolesWatcher(t *testing.T) {

            // This roles instance reads from files
            rw := NewRolesFiles()

            // This is where our external process would place files containing
            // credential data retrieved from AWS.
            rw.BaseDir = "./test_files"

            // The files underneath the BaseDir for the respective credential parts.
            rw.AccessKeyFile = "role_access_key"
            rw.SecretFile = "role_secret_key"
            rw.TokenFile = "role_token"

            // We want to read the credential data once to make sure we can do
            // so correctly in the foreground.
            rw_err := rw.RolesRead()
            if rw_err != nil {
                t.Errorf("roles read err")
            }

            // We are about to launch our file watcher in the background. Once launched,
            // the var `rw` will be considered to always have the newest credential information.

            // `c` is a channel for the RolesWatcher to send errors on
            c := make(chan error)
            // `s` is a channel for the RolesWatcher to tell us when it has refreshed credentials.
            s := make(chan bool)

            go rw.RolesWatch(c, s)

            go func() {
                for {
                    select {
                        // This case tells us the files have been re-read
                        case <-s:
                            fmt.Printf("\n*********** re-read the files\n")
                            accessKey, secret, token, get_err := rw.Get()
                            if get_err != nil {
                                e := fmt.Sprintf("cannot get a role file:%s\n", get_err.Error()
                                t.Errorf(e)
                            } else {
                                fmt.Printf("new data\naccess key:%s\nsecret:%s\ntoken:%s\n",
                                           accessKey, secret, token)
                            }
                        // We want to short-circuit the test after five seconds
                        case <-time.After(5 * time.Second):
                            fmt.Printf("exiting watcher loop\n")
                            c <- nil
                    }
                }
            }()

            // Give the RolesWatcher a second to start watching files.
            time.Sleep(1 * time.Second)
            // Now we want to touch our files to change their timestamp, which should
            // trigger the RolesWatcher to re-read them, which will then cause a message
            // to be sent on the `s` channel above.
            touch_cmd1 := exec.Command("touch",rw.BaseDir + string(filepath.Separator) + rw.AccessKeyFile)
            touch_cmd2 := exec.Command("touch",rw.BaseDir + string(filepath.Separator) + rw.SecretFile)
            touch_cmd3 := exec.Command("touch",rw.BaseDir + string(filepath.Separator) + rw.TokenFile)
            touch_err1 := touch_cmd1.Run()
            if touch_err1 != nil {
                t.Errorf(touch_err1.Error())
            }
            touch_err2 := touch_cmd2.Run()
            if touch_err2 != nil {
                t.Errorf(touch_err2.Error())
            }
            touch_err3 := touch_cmd3.Run()
            if touch_err3 != nil {
                t.Errorf(touch_err3.Error())
            }
            time.Sleep(2 * time.Second)

            // If this is triggered, then it means something went wrong reading the credentials files.
            watch_err := <-c
            if watch_err != nil {
                e := fmt.Sprintf("error from watcher: %s\n", watch_err.Error())
                t.Errorf(e)
            }
        }


### RolesMaster

This instance of the `RolesReader` interface only accepts a one-time initialization of the
AccessKey and Secret. Since "master" credentials do
not contain a Token, it is forced to be empty. This instance will panic if `RolesWatch`
is called, as this instance does not support the concept of online updating.
Using this package
is trivial: instantiate a new instance with `NewRolesMasterCredentials(accessKey, secret)` and
extract the values with the various `Get*` functions.

### RolesSimple

This is the simplest `RolesReader` interface. It is just a safe container for the AccessKey,
Secret and Token. Unlike RolesMaster, a token can be set. This instance will panic if `RolesWatch`
is called, as this instance does not support the concept of online updating. Using this package
is trivial: instantiate a new instance with `NewRolesSimple(accessKey, secret, token)` and
extract the values with the various `Get*` functions.

### Installation

        go get github.com/smugmug/goawsroles/roles_files
        go get github.com/smugmug/goawsroles/roles_master
        go get github.com/smugmug/goawsroles/roles_simple

### Dependencies

This package depends on the nascent "official" AWS SDK for Go at

        github.com/awslabs/aws-sdk-go/aws

This package changes often so you may need to update it directly if you have build issues with goawsroles.

### Contact Us

Please contact <EMAIL> for information related to this package.
Pull requests also welcome!<commit_msg>rm note about depend lib<commit_after>goawsroles
==========

`goawsroles` provides an interface and sample implementations for safe handling of AWS
credentials. While typically used with the `godynamo` package (https://github.com/smugmug/godynamo),
it is useful on its own and should be easy to integrate into any package requiring a
consistent interface for AWS credentials.

Typically, AWS credentials consist of a AccessKey, Secret and optionally a Token if
temporary credentials are used. We want to be able to safely update these and access
them in programs. It is useful to have a variety of mechanisms for sourcing these credentials -
hardcoded, environment variables, or local files. `goawsroles` seeks to provide one interface
for these and other scenarios, with the goal being safe access to read and update credential data.

At SmugMug, our typical use involves local files in `/etc` for each of the AccessKey, Secret
and Token. These are updated by another process every thirty minutes. Local files provide
for ease-of-use with some decent security via user and group permissions that are more
effective than environment-variable based solutions.  The `RolesFiles`
implementation included here provides a solution for this scenario; it includes a
`RolesWatch` method which, when launched via a goroutine, will watch the local
permissions files for updates. When all three files are updated within a sane time
interval, the internal representation of the permission strings for the AccessKey,
Secret and Token will be updated. Using this mechanism along with another process
to keep our local files updated and valid, we can be sure that any time we call
the `Get` method on the `RolesFiles` instance, we are getting fresh credentials.

### RolesFiles Example

This example is contained within the unit test for the `RolesFiles` instance, with
some extra comments.

        func TestRolesWatcher(t *testing.T) {

            // This roles instance reads from files
            rw := NewRolesFiles()

            // This is where our external process would place files containing
            // credential data retrieved from AWS.
            rw.BaseDir = "./test_files"

            // The files underneath the BaseDir for the respective credential parts.
            rw.AccessKeyFile = "role_access_key"
            rw.SecretFile = "role_secret_key"
            rw.TokenFile = "role_token"

            // We want to read the credential data once to make sure we can do
            // so correctly in the foreground.
            rw_err := rw.RolesRead()
            if rw_err != nil {
                t.Errorf("roles read err")
            }

            // We are about to launch our file watcher in the background. Once launched,
            // the var `rw` will be considered to always have the newest credential information.

            // `c` is a channel for the RolesWatcher to send errors on
            c := make(chan error)
            // `s` is a channel for the RolesWatcher to tell us when it has refreshed credentials.
            s := make(chan bool)

            go rw.RolesWatch(c, s)

            go func() {
                for {
                    select {
                        // This case tells us the files have been re-read
                        case <-s:
                            fmt.Printf("\n*********** re-read the files\n")
                            accessKey, secret, token, get_err := rw.Get()
                            if get_err != nil {
                                e := fmt.Sprintf("cannot get a role file:%s\n", get_err.Error()
                                t.Errorf(e)
                            } else {
                                fmt.Printf("new data\naccess key:%s\nsecret:%s\ntoken:%s\n",
                                           accessKey, secret, token)
                            }
                        // We want to short-circuit the test after five seconds
                        case <-time.After(5 * time.Second):
                            fmt.Printf("exiting watcher loop\n")
                            c <- nil
                    }
                }
            }()

            // Give the RolesWatcher a second to start watching files.
            time.Sleep(1 * time.Second)
            // Now we want to touch our files to change their timestamp, which should
            // trigger the RolesWatcher to re-read them, which will then cause a message
            // to be sent on the `s` channel above.
            touch_cmd1 := exec.Command("touch",rw.BaseDir + string(filepath.Separator) + rw.AccessKeyFile)
            touch_cmd2 := exec.Command("touch",rw.BaseDir + string(filepath.Separator) + rw.SecretFile)
            touch_cmd3 := exec.Command("touch",rw.BaseDir + string(filepath.Separator) + rw.TokenFile)
            touch_err1 := touch_cmd1.Run()
            if touch_err1 != nil {
                t.Errorf(touch_err1.Error())
            }
            touch_err2 := touch_cmd2.Run()
            if touch_err2 != nil {
                t.Errorf(touch_err2.Error())
            }
            touch_err3 := touch_cmd3.Run()
            if touch_err3 != nil {
                t.Errorf(touch_err3.Error())
            }
            time.Sleep(2 * time.Second)

            // If this is triggered, then it means something went wrong reading the credentials files.
            watch_err := <-c
            if watch_err != nil {
                e := fmt.Sprintf("error from watcher: %s\n", watch_err.Error())
                t.Errorf(e)
            }
        }


### RolesMaster

This instance of the `RolesReader` interface only accepts a one-time initialization of the
AccessKey and Secret. Since "master" credentials do
not contain a Token, it is forced to be empty. This instance will panic if `RolesWatch`
is called, as this instance does not support the concept of online updating.
Using this package
is trivial: instantiate a new instance with `NewRolesMasterCredentials(accessKey, secret)` and
extract the values with the various `Get*` functions.

### RolesSimple

This is the simplest `RolesReader` interface. It is just a safe container for the AccessKey,
Secret and Token. Unlike RolesMaster, a token can be set. This instance will panic if `RolesWatch`
is called, as this instance does not support the concept of online updating. Using this package
is trivial: instantiate a new instance with `NewRolesSimple(accessKey, secret, token)` and
extract the values with the various `Get*` functions.

### Installation

        go get github.com/smugmug/goawsroles/roles_files
        go get github.com/smugmug/goawsroles/roles_master
        go get github.com/smugmug/goawsroles/roles_simple

### Contact Us

Please contact <EMAIL> for information related to this package.
Pull requests also welcome!
{'fraction_non_alphanumeric': 0.07032490974729241, 'fraction_numerical': 0.00259927797833935, 'mean_word_length': 2.149386084583902, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6998465', 'n_tokens_mistral': 3514, 'n_tokens_neox': 3400, 'n_words': 1502}