<jupyter_start><jupyter_text>OverviewThis demo walks through connecting to Ceph from an EPIC cluster using python (not Spark).  Setup- On your client machine, run the script `./scripts/end_user_scripts/ceph/1_demo_server_setup.sh` to setup a ceph nano server on the RDP Server host- Add the EPIC Spark 2.4 image- Configure EPIC with Active Directory [see README](https://github.com/bluedata-community/bluedata-demo-env-aws-terraform/blob/master/docs/README-AD.md)- Setup Demo Tenant with Active Directory [see README](https://github.com/bluedata-community/bluedata-demo-env-aws-terraform/blob/master/docs/README-AD.md)- Provision a Spark cluster in the Demo Tenant with:  - 1 x Spark Controller (small)  - 1 x Jupyter Hub (small)  Connect - Verify that we are able to get a response from the ceph instance. We should see something like:```anonymous```<jupyter_code>! curl 10.1.0.216:8080 # Change to the private IP of RDP server<jupyter_output><empty_output><jupyter_text>- Install the boto library<jupyter_code>! pip install --user boto # now restart kernel to pick up the boto library<jupyter_output>Requirement already satisfied: boto in ./.local/lib/python3.6/site-packages (2.49.0)<jupyter_text>- Set up the connection<jupyter_code>import boto
import boto.s3.connection
access_key = 'sandboxAccessKey'
secret_key = 'sandboxSecretKey'
host       = '10.1.0.216' # Change to the private IP of RDP server 

conn = boto.connect_s3(
        aws_access_key_id = access_key,
        aws_secret_access_key = secret_key,
        host = host,  
        port = 8080,
        is_secure=False,
        calling_format = boto.s3.connection.OrdinaryCallingFormat(),
        )<jupyter_output><empty_output><jupyter_text>- List the current buckets<jupyter_code>for bucket in conn.get_all_buckets():
        print("{name}\t{created}".format(
                name = bucket.name,
                created = bucket.creation_date,
        ))<jupyter_output>sandboxbucket	2020-04-23T21:38:01.127Z<jupyter_text>- Create a new bucket<jupyter_code>bucket = conn.create_bucket('my-new-bucket')<jupyter_output><empty_output><jupyter_text>- List the buckets<jupyter_code>for bucket in conn.get_all_buckets():
        print("{name}\t{created}".format(
                name = bucket.name,
                created = bucket.creation_date,
        ))
bucket = conn.delete_bucket('my-new-bucket')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10488926034266612, 'fraction_numerical': 0.0221479314667781, 'mean_word_length': 3.997912317327766, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '542619', 'license': 'Apache-2.0', 'n_tokens_mistral': 835, 'n_tokens_neox': 778, 'n_words': 194, 'path': 'docs/README-CONNECT-TO-CEPH-FROM-PYTHON-NB.ipynb', 'repo_name': 'jaideepjoshi/hcp-demo-env-aws-terraform', 'size': '2393'}