<jupyter_start><jupyter_text>How do I copy files to a folder within a project OverviewFiles within a project can be placed within a project using an advanced access feature in the API (folders):Here we focus on creating a folder and moving a file from your project into that folder. WarningThis is a SBG Advanced Access Feature and is in beta. ImportsWe import the Api class from the official sevenbridges-python bindings below.. Initialize the objectThe Api object needs to know your auth_token and the correct path. Here we assume you are using the .sbgrc file in your home directory. For other options see Setup_API_environment.ipynb<jupyter_code>import sevenbridges as sbg

#Authentication
config_config_file = sbg.Config(profile='cgc')
api = sbg.Api(config=config_config_file)
print(api.users.me())<jupyter_output><empty_output><jupyter_text>Create a new projectWe start by listing all of your projects and your billing groups. Next we create the JSON that will be passed to the API to create the project. The dictionary should include:* billing_group Billing group that will be charged for this project* description (optional) Project description* name Name of the project, may be non-unique* tags List of tags, set tags[0] = 'tcga' if you will use protected dataAfter creating the project, you can re-check the project list and get additional details assigned by the CGC, including:* id Unique identifier for the project, generated based on Project Name* href Address of the project.* type (unimportant) this is always equal to 'v2'<jupyter_code># [USER INPUT] Set project name here:
new_project_name = 'Fold-away'                          
      
    
# What are my funding sources?
billing_groups = api.billing_groups.query()  

# Pick the first group (arbitrary)
print((billing_groups[0].name + \
       ' will be charged for computation and storage (if applicable) for your new project'))

# Set up the information for your new project
new_project = {
        'billing_group': billing_groups[0].id,
        'description': """A project created by the API recipe (projects_makeNew.ipynb).
                      This also supports **markdown**
                      _Pretty cool_, right?
                   """,
        'name': new_project_name
}

# check if this project already exists. LIST all projects and check for name match
my_project = [p for p in api.projects.query(limit=100).all() \
              if p.name == new_project_name]      
              
if my_project:    # exploit fact that empty list is False, {list, tuple, etc} is True
    print('A project with the name (%s) already exists, please choose a unique name' \
          % new_project_name)
    raise KeyboardInterrupt
else:
    # CREATE the new project
    my_project = api.projects.create(name = new_project['name'], \
                                     billing_group = new_project['billing_group'], \
                                     description = new_project['description'])
    
    # (re)list all projects, and get your new project
    my_project = [p for p in api.projects.query(limit=100).all() \
              if p.name == new_project_name][0]<jupyter_output><empty_output><jupyter_text>Copy a file from the Public ReferenceWe will first find our _source_project_ (the Public Reference Files), then list the files within the source project1, and copy a file from _source_project_ -> _my_project_.The critical information for this POST is the file_id. Note, you are allow to copy the same file as many times as you like. However, duplicates will be automatically have a prefix attached of (_1_, _2_, etc) depending on how many times you copy the file.<jupyter_code>file_names = ["G20479.HCC1143.2.converted.pe_1_1Mreads.fastq", \
  "G20479.HCC1143.2.converted.pe_2_1Mreads.fastq"]
#Copying from public reference files
source_project_id = 'admin/sbg-public-data'  
source_files = list(api.files.query(limit = 100, project = source_project_id).all())
reqd_input_files = [currFile for currFile in source_files if currFile.name in file_names]
for currFile in reqd_input_files:
    my_new_file = currFile.copy(project = my_project.id, name = currFile.name)
    print(my_new_file.name + "\t" + my_new_file.id)
    if "fastq" in my_new_file.name:
        my_new_file.tags = ["Reads", "Demo"]
        my_new_file.save()<jupyter_output><empty_output><jupyter_text>Function to send an API call and receive its resultsBelow, we define a simple function to send and receive JSONs from the API using the correctly formatted HTTP calls. The necessary imports are handled above. In addition, we initialize the authentication token.<jupyter_code>auth_token = 'Insert Authentication Token'
import json
from requests import request

def api_call(path, method='GET', query=None, data=None, token=None):
     
    base_url = 'https://cgc-api.sbgenomics.com/v2/'
     
    data = json.dumps(data) if isinstance(data, dict) \
    or isinstance(data,list) else None
               
    headers = {
        'X-SBG-Auth-Token': token,
        'Accept': 'application/json',
        'Content-type': 'application/json',
        'X-SBG-advance-access': 'advance'
    }
    
    response = request(method, base_url + path, params=query, \
                       data=data, headers=headers)
    try:
        response_dict = response.json()
    except:
        response_dict = {}
    if int(response.status_code / 100) != 2:
        raise Exception('Server responded with status code %s.' % response.status_code)
    return response_dict<jupyter_output><empty_output><jupyter_text>Create a folderWe need to create the JSON object with the information:* name: name of the folder, required* type: folder to indicate that a folder is being created, required* project: project in which the folder is being created.* parent: parent is required if a folder is created within another folder. Then parent should be the id of the folder under which the new folder is to be created.Either one of Project or Parent needs to be created.The API returns the following information:* href: Address of the fiolder.* id: id of the folder.* modified_on: Date on which the folder was modified last (upto seconds).* created_on: Date on which the folder was created (upto seconds).* type: folder or file * name: name of the folder* parent: project specific ID* project: project ID.<jupyter_code>new_folder_body = {
  "name": "reads", 
  "project": my_project.id,
  "type": "FOLDER"
}
print(new_folder_body)
new_folder = api_call(path='files', method='POST', data=new_folder_body, token=auth_token)
print(new_folder)<jupyter_output><empty_output><jupyter_text>Lisiting files and folder within the projectGetting a list of the files and folder within the project requires advanced access in the API. So, we cannot use the python bindings for this API call.<jupyter_code>my_files = api_call(path="files?project=" + my_project.id, method="GET", token=auth_token)
print("Name \t id \t parent \t type")
for curr_file in my_files['items']:
    print(curr_file["name"] + "\t" + curr_file["id"] + "\t" + curr_file["parent"] + "\t" + curr_file["type"])<jupyter_output><empty_output><jupyter_text>Moving the files to the folderGet the file objects that need to be moved to the folder and then move the files to the folder<jupyter_code>my_files = api.files.query(project=my_project.id, names=["G20479.HCC1143.2.converted.pe_1_1Mreads.fastq", "G20479.HCC1143.2.converted.pe_2_1Mreads.fastq"])

for curr_file in my_files:
    folder_body = {
      "parent": new_folder["id"], 
      "name": curr_file.name
    }
    api_call(path='files/' + curr_file.id + "/actions/move", method='POST', data=folder_body, token=auth_token)
my_files = api_call(path="files/" + new_folder["id"] + "/list", method="GET", token=auth_token)
print("Files within the folder are:")
for curr_file in my_files['items']:
    print(curr_file["name"] + "\t" + curr_file["id"] + "\t" + curr_file["parent"] + "\t" + curr_file["type"])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09310214851111949, 'fraction_numerical': 0.008795074758135445, 'mean_word_length': 3.947172156619018, 'pattern_counts': {'":': 5, '<': 29, '<?xml version=': 0, '>': 30, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '478760', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2361, 'n_tokens_neox': 2309, 'n_words': 847, 'path': 'Recipes/CGC/files_moveToFolder.ipynb', 'repo_name': 'sbg/okAPI', 'size': '7959'}