<jupyter_start><jupyter_text>pyOpenCGA Basic User Usage------**[NOTE]** The server methods used by pyopencga client are defined in the following swagger URL:- http://bioinfo.hpc.cam.ac.uk/opencga-demo/webservicesFor tutorials and more info about accessing the OpenCGA REST please read the documentation at http://docs.opencb.org/display/opencga/Python  Loading pyOpenCGA<jupyter_code># Initialize PYTHONPATH for pyopencga
import sys
import os
from pprint import pprint<jupyter_output><empty_output><jupyter_text>Now is time to import pyopencga modules.You have two optionsa) You can import pyopencga directly (skip next section) if you have installed pyopencga with `pip install pyopencga`  (remember to use  `sudo` unless you are using your own Python install or virtualenv)b) If you need to import from the source code remember that Python3 does not accept relative importing, so you need to append the module path to `sys.path`  Preparing environmnet for importing from source<jupyter_code>cwd = os.getcwd()
print("current_dir: ...."+cwd[-10:])

base_modules_dir = os.path.dirname(cwd)
print("base_modules_dir: ...."+base_modules_dir[-10:])

sys.path.append(base_modules_dir)<jupyter_output>current_dir: ..../notebooks
base_modules_dir: ..../pyOpenCGA<jupyter_text>Importing pyopencga<jupyter_code>from pyopencga.opencga_config import ClientConfiguration 
from pyopencga.opencga_client import OpenCGAClient
import json<jupyter_output><empty_output><jupyter_text>Creating some useful functions to manage the results<jupyter_code>def get_not_private_methods(client):
    all_methods = dir(client)
    
    #showing all methos (exept the ones starting with "_", as they are private for the API)
    methods = [method for method in all_methods if not method.startswith("_")]
    return methods<jupyter_output><empty_output><jupyter_text>Setup client and login **Configuration and Credentials** You need to provide a server URL in the standard configuration format for OpenCGA as a dict or in a json fileRegarding credentials, if you don't pass the password, it would be asked interactively without echo.<jupyter_code># server host

# user credentials
user = "demo"
passwd = "<PASSWORD>"

# the user demo access projects from user opencga
prj_owner = "opencga"<jupyter_output><empty_output><jupyter_text>Creating ConfigClient for server connection configuration<jupyter_code># Creating ClientConfiguration dict
host = 'http://bioinfo.hpc.cam.ac.uk/opencga-demo'

config_dict = {"rest": {
                       "host": host 
                    }
               }

print("Config information:\n",config_dict)<jupyter_output>Config information:
 {'rest': {'host': 'http://bioinfo.hpc.cam.ac.uk/opencga-demo'}}<jupyter_text>Initialize the client configurationYou can pass a dictionary to the ClientConfiguration<jupyter_code>config = ClientConfiguration(config_dict)
oc = OpenCGAClient(config)<jupyter_output><empty_output><jupyter_text>Make the login<jupyter_code># here we put only the user in order to be asked for the password interactively
oc.login(user)<jupyter_output>····<jupyter_text>**You are now connected to OpenCGA**  Working with Users<jupyter_code># Listing available methods for the user client object
user_client = oc.users

# showing all methods (except the ones starting with "_", as they are private for the API)
get_not_private_methods(user_client)
## getting user information
## [NOTE] User needs the quey_id string directly --> (user)
uc_info = user_client.info(user).responses[0]['results'][0]

print("user info:")
print("name: {}\towned_projects: {}".format(uc_info["name"], len(uc_info["projects"])))<jupyter_output>user info:
name: Demo	owned_projects: 0<jupyter_text>The demo user has not projects from its own, but has access to some projectso from `opencga` user.Let's see how to find it out.We need to list the project info from project client not from the user client.We use the method `search()`And remember that OpenCGA REST objects encapsulate the result inside the responses property, so we need to access the first element of the responses array.<jupyter_code>## Getting user projects
## [NOTE] Client specific methods have the query_id as a key:value (i.e (user=user_id)) 
project_client = oc.projects
projects_info = project_client.search().responses[0]["results"]

for project in projects_info:
    print("Name: {}\tfull_id: {}".format(project["name"], project["fqn"]))<jupyter_output>Name: Exomes GRCh37	full_id: opencga@exomes_grch37<jupyter_text>**User demo has access to one project called opencga@exomes_grch37**note: in opencga the projects and studies have a `full qualify name, fqn` with the format [owner]@[porject]:[study]   Working with Projects<jupyter_code>project_client = oc.projects

get_not_private_methods(project_client)
## Getting all projects from logged in user
project_client = oc.projects
projects_list = project_client.search().responses[0]["results"]

for project in projects_list:
    print("Name: {}\tfull_id: {}".format(project["name"], project["fqn"]))
## Getting information from a specific project
project_name = 'exomes_grch37'
project_info = project_client.info(project_name).responses[0]['results'][0]

#show the studies
for study in project_info['studies']:
    print("project:{}\nstudy:{}\ttype:{}".format(project_name, study['name'], study['type'] ))
    print('--')
## Fetching the studies from a project using the studies method
results = project_client.studies(project_name).responses[0]['results']
for result in results:
    pprint(result)<jupyter_output>{'attributes': {},
 'cipher': 'none',
 'cohorts': [],
 'creationDate': '20190604154741',
 'dataStores': {},
 'datasets': [],
 'description': '',
 'experiments': [],
 'files': [],
 'fqn': 'opencga@exomes_grch37:corpasome',
 'groups': [{'id': '@members',
             'name': '@members',
             'userIds': ['opencga', 'demo']},
            {'id': '@admins', 'name': '@admins', 'userIds': []}],
 'id': 'corpasome',
 'individuals': [],
 'jobs': [],
 'lastModified': '20190604154741',
 'modificationDate': '20190604154741',
 'name': 'Corpasome',
 'panels': [],
 'permissionRules': {},
 'release': 1,
 'samples': [],
 'size': 0,
 'stats': {},
 'status': {'date': '20190604154741', 'message': '', 'name': 'READY'},
 'type': 'CASE_CONTROL',
 'uri': 'file:///mnt/data/opencga-demo/sessions/users/opencga/projects/1/2/',
 'uuid': 'Iyy1cwFrAAIAAViPXu86gw',
 'variableSets': []}
{'attributes': {},
 'cipher': 'none',
 'cohorts': [],
 'creationDate': '20190617155526',
 'dataStores': {},
 'datasets': [],
 '[...]
{'fraction_non_alphanumeric': 0.13018204069144867, 'fraction_numerical': 0.01529753709652746, 'mean_word_length': 5.782157676348548, 'pattern_counts': {'":': 2, '<': 40, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '344071', 'license': 'Apache-2.0', 'n_tokens_mistral': 2148, 'n_tokens_neox': 2043, 'n_words': 593, 'path': 'opencga-client/src/main/python/notebooks/pyopencga_basic_notebook_001.ipynb', 'repo_name': 'fabbondanza/opencga', 'size': '6531'}