<jupyter_start><jupyter_text>How do I write files to my own _cloud storage_? OverviewConnecting to cloud storage facilitates sharing and leveraging bioinformatics data that is commonly stored there (details [here](http://docs.sevenbridges.com/docs/connecting-cloud-storage-overview)). Here, we show an example of **moving** files from the _Seven Bridges Platform_ to a user's _Amazon Web Services_ **bucket**. Afterwards, an **alias** to the file will be preserved in the user's _Seven Bridges Platform_ project. Prerequisites 1. You need your _authentication token_ and the API needs to know about it. See **Setup_API_environment.ipynb** for details. 2. You need to have an AWS account with an S3 bucket 3. You need to be a member (or owner) of a project that has some files in it.  Setting up Identity and Access Management (IAM) access There is a detailed tutorial to set up access to your buckets on [AWS](http://docs.sevenbridges.com/v1.0/docs/aws-cloud-storage-tutorial) or [GCP](https://docs.sevenbridges.com/docs/google-cloud-storage-gcs-volumes).  ImportsWe import the _Api_ class from the official sevenbridges-python bindings below.<jupyter_code>import sevenbridges as sbg<jupyter_output><empty_output><jupyter_text>Initialize the objectThe `Api` object needs to know your **auth\_token** and the correct path. Here we assume you are using the credentials file in your home directory. For other options see Setup_API_environment.ipynb<jupyter_code># [USER INPUT] specify credentials file profile {cgc, sbg, default}
prof = 'default'

config_file = sbg.Config(profile=prof)
api = sbg.Api(config=config_file)<jupyter_output><empty_output><jupyter_text>Mount the volumeSince we have configured the IAM policy in the prior step, we can now mount the volume. Make sure to **replace** (BUCKET-NAME, INSERT ACCESS KEY ID, and INSERT SECRET ACCESS KEY) below WarningYou only need to mount a volume **once**. If you have already done this, then skip this cell and go to _Get details of the volume_. If you try to mount it more than once, you will get an error (likely error: `Forbidden: Could not obtain cross-write access on the service. Check the documentation.`). Additional information * Detailed documentation of this particular REST architectural style request is available [here](http://docs.sevenbridges.com/docs/create-a-volume-v2)<jupyter_code>volume_import = api.volumes.create_s3_volume(
    name='my_volume_RW', 
    bucket='BUCKET-NAME',
    access_key_id='INSERT ACCESS KEY ID',
    secret_access_key = 'INSERT SECRET ACCESS KEY',
    access_mode='RW')<jupyter_output><empty_output><jupyter_text>Get details of the volumeLet's list all our volumes and get one volume's (`my_volume_RW`, which we have just imported in the prior cell) details to make sure it's mounted as we expected. Additional information * Detailed documentation of these particular REST architectural style request are available [here (list)](http://docs.sevenbridges.com/docs/list-volumes-v2) and [here (details)](http://docs.sevenbridges.com/docs/get-details-of-a-volume-v2)<jupyter_code># [USER INPUT] Set volume name here:
volume_name = 'my_volume'  


all_volumes = api.volumes.query()

my_volume = [v for v in all_volumes.all() 
            if v.name == volume_name]

if not my_volume:
    print('Volume {} does not exist, check name / mounting'
          .format(volume_name))
    raise KeyboardInterrupt
else:
    my_volume = my_volume[0]

print(my_volume.name)<jupyter_output><empty_output><jupyter_text>Get some files to moveHere we pick a project and a few files to move. In case you don't have any files, you could add them via the [GUI](http://docs.sevenbridges.com/docs/copy-files-using-the-visual-interface) or an API [recipe](files_copyFromMyProject.ipynb).  PROTIPS * Using this [recipe](files_upload_and_setMetadata.ipynb) to upload some files would mean **you don't need to change** names in the _file\_list_ below (if this is the first time you are running this script). * A detailed _recipe_ for listing projects is [here](projects_listAll.ipynb) * A detailed _recipe_ for listing files is [here](files_listAll.ipynb)<jupyter_code># [USER INPUT] Set project name here:
project_id = 'division-name/project-name'
file_list = ['files_listAll.ipynb',
            'files_copyFromMyProject.ipynb',
            'files_copyFromPublicReference.ipynb']

    
# Find files in your project matching your file_list
my_files = api.files.query(limit=100, project=project_id,
                          names=file_list)<jupyter_output><empty_output><jupyter_text>Start an export jobSpin up a job to export files from the _Seven Bridges Platform_ to your _Cloud Storage_ (Volume). Additional information * Detailed documentation of these particular REST architectural style request are available [here](https://docs.sevenbridges.com/referencestart-a-bulk-export-job)  Possible ERRORSNote that you are only permitted to export a file _we host for you_. This **does not include** **Public Reference** files (even if you have _copied_ them into your project), other **imported files** (aliases) or **archived files** that have not been restored.The error you would likely see for doing this is `Forbidden: Requested file cannot be exported. Check the documentation.`<jupyter_code># Loop through selected files and submit a bulk export request
exports = []

for f in my_files:
    export = {
            'file':f,
            'volume': my_volume,
            'location':f.name
    }
    exports.append(export)

export_jobs = api.exports.bulk_submit(exports=exports, copy_only=True)

errors = []
for bulk_item, job in zip(exports, export_jobs):
    if job.error:
        error_message = "File: %s - %s" % (
            bulk_item['file'].name, job.error.message)
        errors.append(error_message)
if errors:
    print(
        'There were errors with bulk export submission:',
        '"; "'.join(errors)
    )
    

# Monitor export jobs where no errors were encountered
submitted_jobs = [job.resource for job in export_jobs if job.resource]<jupyter_output><empty_output><jupyter_text>List and detail export jobsThe `api.exports.bulk_get` method will capture details of all exports which have been sent in the `bulk_submit` call. Additional information * Detailed documentation of these particular REST architectural style request are available [here](http://docs.sevenbridges.com/docs/list-export-jobs-v2) and [here](https://docs.sevenbridges.com/referenceget-details-of-a-bulk-export-job)<jupyter_code>export_jobs = api.exports.bulk_get(exports=[s.id for s in submitted_jobs])

for j in export_jobs:
    print('File {} export stated {}; it is {}'
         .format(j.resource.destination, j.resource.started_on, j.resource.state))
print('\n')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09112426035502959, 'fraction_numerical': 0.0022189349112426036, 'mean_word_length': 4.676742233417296, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '469950', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2009, 'n_tokens_neox': 1940, 'n_words': 714, 'path': 'Recipes/SBPLAT/volumes_writeToCloudStorage.ipynb', 'repo_name': 'sbg/okAPI', 'size': '6760'}