<jupyter_start><jupyter_text>Cromwell-tools Python API QuickstartThis notebook will help you get familiar with the Cromwell-tools' Python API client. After walking through this notebook you should be able to import cromwell-tools in your Python application and talk to Cromwell engine using various authentication methods.  Standard Authentication  1. [Recommended] Authenticate with Cromwell using the standardized methodcromwell-tools provides a standard method `cromwell_tools.cromwell_auth.CromwellAuth.harmonize_credentials` to authenticate with various Cromwell instances, which is the preferred auth method.<jupyter_code>from cromwell_tools.cromwell_auth import CromwellAuth

# Authenticate with Cromwell using HTTPBasicAuth (username and password)
auth = CromwellAuth.harmonize_credentials(username='username', 
                                          password='password', 
                                          url='https://cromwell.xxx.broadinstitute.org')

# Authenticate with Cromwell using HTTPBasicAuth (secret JSON file)
auth_2 = CromwellAuth.harmonize_credentials(secrets_file='path/to/secrets_file.json')

# Authenticate with Cromwell using OAuth (service account JSON key file)
auth_3 = CromwellAuth.harmonize_credentials(service_account_key='path/to/service/account/key.json', 
                                            url='https://cromwell.caas-prod.broadinstitute.org')

# Authenticate with Cromwell with no Auth
auth_4 = CromwellAuth.harmonize_credentials(url='https://cromwell.caas-prod.broadinstitute.org')<jupyter_output><empty_output><jupyter_text>2. Authenticate with Cromwell using HTTPBasicAuth (username and password)<jupyter_code>from cromwell_tools.cromwell_auth import CromwellAuth

auth_httpbasic = CromwellAuth.from_user_password(username='username', 
                                                 password='password', 
                                                 url='https://cromwell.xxx.broadinstitute.org')<jupyter_output><empty_output><jupyter_text>3. Authenticate with Cromwell using HTTPBasicAuth (secret JSON file) A secret JSON file should look like the following example with 3 keys:```JSON{    "url": "url",    "username": "username",    "password": "password"}```<jupyter_code>from cromwell_tools.cromwell_auth import CromwellAuth

auth_httpbasic_file = CromwellAuth.from_secrets_file(secrets_file='path/to/secrets_file.json')<jupyter_output><empty_output><jupyter_text>4. Authenticate with Cromwell using OAuth (service account JSON key file)<jupyter_code>from cromwell_tools.cromwell_auth import CromwellAuth

auth_oauth = CromwellAuth.from_service_account_key_file(service_account_key='path/to/service/account/key.json', 
                                                        url='https://cromwell.caas-prod.broadinstitute.org')<jupyter_output><empty_output><jupyter_text>5. Authenticate with Cromwell with no Auth<jupyter_code>from cromwell_tools.cromwell_auth import CromwellAuth

auth_no_auth = CromwellAuth.from_no_authentication(url='https://cromwell.caas-prod.broadinstitute.org')<jupyter_output><empty_output><jupyter_text>Submit jobs to Cromwell/Cromwell-as-a-Service<jupyter_code>from cromwell_tools import api
from cromwell_tools import utilities
response = api.submit(auth=auth, 
                      wdl_file='Examples/hello_world.wdl', 
                      inputs_files=['Examples/inputs.json'], 
                      dependencies=['Examples/helloworld.wdl'])
response.json()
api.wait(workflow_ids=[response.json()['id']], auth=auth, poll_interval_seconds=3)<jupyter_output>All workflows succeeded!<jupyter_text>Query jobs in Cromwell/Cromwell-as-a-Service  Get the workflows<jupyter_code>from cromwell_tools import api
from cromwell_tools import utilities<jupyter_output><empty_output><jupyter_text>There are a lot of query keys can be used to filter workflowsA complicated query dict could be:```pythoncustom_query_dict = {    'label': {        'cromwell-workflow-id': 'cromwell-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',        'project_shortname': 'Name of a project that trigerred this workflow'    },    'status': ['Running', 'Succeeded', 'Aborted', 'Submitted', 'On Hold', 'Failed'],    'id': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',    'additionalQueryResultFields': 'labels',    'submission': '2018-01-01T00:01:01.410150Z',    'start': '2018-01-01T01:01:01.410150Z',    'end': '2018-01-01T02:01:01.410150Z',    'name': ['SmartSeq2SingleCell', '10xCount'],    'additionalQueryResultFields': ['labels'],    'includeSubworkflows': True}```We will just use a very simple example here to query for the workflow we just submitted:<jupyter_code>custom_query_dict = {
    'name': 'HelloWorld',
    'id': response.json()['id']
}

response = api.query(query_dict=custom_query_dict, auth=auth)
response.json()['results']<jupyter_output><empty_output><jupyter_text>Submit jobs with On Hold in Cromwell/Cromwell-as-a-Service- Submit a job with "On Hold" status.- Check the status.- Release the job.  Submit the job<jupyter_code>from cromwell_tools import api
from cromwell_tools import utilities
response = api.submit(auth=auth, 
                      wdl_file='Examples/hello_world.wdl', 
                      inputs_files=['Examples/inputs.json'], 
                      dependencies=['Examples/helloworld.wdl'], 
                      on_hold=True)
response.json()<jupyter_output><empty_output><jupyter_text>Check the status of the job<jupyter_code>response = api.status(auth=auth, 
                      uuid=response.json()['id'])
response.json()<jupyter_output><empty_output><jupyter_text>Release the job<jupyter_code>response = api.release_hold(auth=auth, 
                            uuid=response.json()['id'])
response.json()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1023855127981891, 'fraction_numerical': 0.012711126588890823, 'mean_word_length': 4.245662100456621, 'pattern_counts': {'":': 3, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 4, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '123776', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1824, 'n_tokens_neox': 1684, 'n_words': 350, 'path': 'docs/Tutorials/Quickstart/api_quickstart.ipynb', 'repo_name': 'kgalens/cromwell-tools', 'size': '5743'}