<jupyter_start><jupyter_text>Overview------This notebook is intended to provide guidance on performing advanced common administrator actions required for maintaining a OpenCGA installation, from granting permission to other users to updating the metadata for samples or individuals in the database.Before running this notebook, it is highly recommended to check the official OpenCGA documentation about [Sharing and Permissions](https://app.gitbook.com/@opencb/s/opencga/~/drafts/-Magktiifb08PPiPnoYk/manual/data-management/sharing-and-permissions), [Adding Custom Metadata](https://app.gitbook.com/@opencb/s/opencga/~/drafts/-MbG9-KDgXf7z8BZJCXq/manual/data-management/adding-custom-metadata) and [Data Models](https://app.gitbook.com/@opencb/s/opencga/~/drafts/-MbG9-KDgXf7z8BZJCXq/manual/data-models).**[NOTES]** - For guidance on how to login and get started with *pyopencga* you can refer to : [pyopencga_first_steps.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)- A good first step when start working with OpenCGA is to explore **Catalog** the OpenCGA component which holds information about our user, the projects and studies our user has permission to access and the clinical data from the studies. For guidance you can refer to : [pyopencga_catalog.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)- The server methods used by *pyopencga* client are defined in the following swagger URL: https://ws.opencb.org/opencga-prod/webservices/  Table of Contents:This Notebook is organised in the following sections:* [Permission Management](Permission-management)* [Assign Samples to Individuals](Asign-Samples-to-Individuals)* [Define phenotypes or diseases for Individuals](Define-phenotypes-or-disorders-for-Individuals)* [Define, add, edit and remove variable sets](Define,-add,-edit-and-remove-variable-sets)* [Use Cases](Use-Cases)  Setup the Client and Login into *pyopencga*  Configuration and CredentialsLet's assume we already have *pyopencga* installed in our python setup (all the steps described on [pyopencga_first_steps.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)).You need to provide **at least** a host server URL in the standard configuration format for OpenCGA as a python dictionary or in a json file.<jupyter_code>## Step 1. Import pyopencga dependecies
from pyopencga.opencga_config import ClientConfiguration # import configuration module
from pyopencga.opencga_client import OpencgaClient # import client module
from pprint import pprint
from IPython.display import JSON
import json
import matplotlib.pyplot as plt
import seaborn as sns
import datetime

## Step 2. OpenCGA host
host = 'https://ws.opencb.org/opencga-prod'

## Step 3. Create the ClientConfiguration dict
config_dict = {'rest': {
                       'host': host 
                    }
               }

## Step 4. Create the ClientConfiguration and OpenCGA client
config = ClientConfiguration(config_dict)<jupyter_output><empty_output><jupyter_text>Working with two clients: **`demo-admin`** and **`demo-guest`** For the purpose of the training, we will work with two users: **`demo-admin`** (which belongs to the `@admins` group, and thus it has admin priviledges) and **`demo-guest`** (which is not admin of the study). __[NOTE]:__ Working with both users is required for following up the queries used in this notebook.<jupyter_code>## Step 5.1 Define admin user credentials
admin_user = 'demo-admin'
password = '<PASSWORD>'
admin_client = OpencgaClient(config)
admin_client.login(admin_user, password)
print('Logged succesfuly as {} to {}, your token is: {} well done!'.format(admin_user, host, admin_client.token))
## Step 5.2 Define guest user credentials
guest_user = 'demo-guest'
password = '<PASSWORD>'
guest_client = OpencgaClient(config)
guest_client.login(guest_user, password)
print('Logged succesfuly as {} to {}, your token is: {} well done!'.format(guest_user, host, guest_client.token))<jupyter_output>Logged succesfuly as demo-guest to https://ws.opencb.org/opencga-prod, your token is: eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJkZW1vLWd1ZXN0IiwiYXVkIjoiT3BlbkNHQSB1c2VycyIsImlhdCI6MTYyMjcxNzA3MCwiZXhwIjoxNjIyNzIwNjcwfQ.AP1xeBZbGTUQ6JdE0pWGksLbmm8w-LzhBPpN0r4LmPw well done!<jupyter_text>Setup OpenCGA VariablesOnce we have defined a variable with the client configuration and credentials, we can access to all the methods defined for the client. These methods implement calls to query different data models in *OpenCGA*. - For this training we have created a test project and study. As an administrator, we assume that you're familiar with the concept of the `fqn`. If not, please check the documentation [here]().<jupyter_code>## Define the study we will work with
study = 'demo@training:admin'

## Define the user ids from some NOT admin demo users 
user1 = 'trainee1'
user2 = 'trainee2'<jupyter_output><empty_output><jupyter_text>Permission Management------ In OpenCGA all the permissions are established at the study level. For this part of the notebook, it is really handy to take a look at the documentation about [Sharing and Permissions](https://app.gitbook.com/@opencb/s/opencga/manual/data-management/sharing-and-permissions) in OpenCGA.All the studies come intrinsically with two administrative groups: `@admins` and `@members`.- All the users added to a new study belong to the `@members` group. This is useful for keeping track of the users that have access to that specific study. However, `@members` doesn't have any permission defined by default.- Users belonging to the group `@admins`, on the other hand, have administrator priviledges. The admins can add users to an study and grant them permissions. However, an admin can't provide admin priviledges to other users (only the user `owner` of the study is able to do so).  1. Add/remove users from a Study - Add usersFirst, we're going to add the users `trainee1` and `trainee2` to the study. Internally, this means that we are adding those users to the `@members` group of the study.The `studies.update_users()` function has a parameter `action`, which by default is set to **ADD**.<jupyter_code>## Add users to study
update_users = admin_client.studies.update_users(study=study, group='members', data={"users": ["trainee1", "trainee2"]})
update_users.print_results(fields='', metadata=False, title='Groups in Study {}'.format(study))

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(update_users.get_results())<jupyter_output>Groups in Study demo@training:admin
----------------------------------------
#id	userIds
@members	trainee1,demo,imedina,pfurio,demo-admin,trainee2,llopez,jcoll,wspooner<jupyter_text>- Remove usersInversely, we can remove the users `trainee1` and `trainee2` from the study. Internally, this means that we are removing those users from the `@members` group of the study.In this case, we need to set the parameter `action`, (which by default is **ADD**) to **REMOVE**.<jupyter_code>## Remove users from members group of study
remove_users = admin_client.studies.update_users(study=study, group='members', action='REMOVE', data={"users": ["trainee1", "trainee2"]})
remove_users.print_results(fields='', metadata=False, title='Groups in Study {}'.format(study))

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(remove_users.get_results())<jupyter_output><empty_output><jupyter_text>2. Create a group and asign permissionsThe defaul group `@members` does not have any permission for the study. This group only defines which users belong to the study.OpenCGA allows to create admin-defined groups of users. Then, one can grant permissions to this group and assign users. The users in the new group will inherit the permissions granted at a group level.  - Create a new `training_group`We can create a new group `training_group` and add the user `trainee1` to the group. The `studies.update_groups()` function has a parameter `action`, which by default is set to **ADD**.<jupyter_code>## Define the python dict with the name of the group and the users
group_data = {
  "id": "training_group",
  "users": [
    "trainee1"
  ]}

## Define a new Training group.
update_groups = admin_client.studies.update_groups(study=study, action='ADD', data=group_data)
update_groups.print_results(fields='', metadata=False, title='Info about groups')

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(update_groups.get_results())<jupyter_output><empty_output><jupyter_text>- Grant permissions to the group `training_group`For granting permissions, we are going to use the template `view_only`. The permissions included in `view_only` are: The member (user or group) will be given full READ permissions to all the entities (samples, individuals, files etc.).<jupyter_code>## Define the python dict.
permission_data = {
  "study": study,
  "template": "view_only"}

## Update the permissions
update_acl = admin_client.studies.update_acl(study=study, members='@training_group', action='ADD', data=permission_data)
update_acl.print_results(metadata=False, title='Info about permissions for new group')

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(update_acl.get_results())<jupyter_output>Info about permissions for new group
-----------------------------------------
#@training_group
DOWNLOAD_FILES,VIEW_COHORTS,VIEW_AGGREGATED_VARIANTS,VIEW_FAMILIES,VIEW_CLINICAL_ANALYSIS,VIEW_SAMPLES,VIEW_FILES,VIEW_FILE_HEADER,VIEW_COHORT_ANNOTATIONS,VIEW_FAMILY_ANNOTATIONS,VIEW_INDIVIDUALS,VIEW_INDIVIDUAL_ANNOTATIONS,VIEW_FILE_CONTENT,VIEW_JOBS,VIEW_SAMPLE_ANNOTATIONS,VIEW_SAMPLE_VARIANTS,VIEW_PANELS<jupyter_text>- Remove `training_group`Don't forget to remove the group before running the cells above. Otherwise you will get an error because the group that you're trying to create already exists.<jupyter_code>## Define the python dict with the name of the group and the users
# group_data = {
#   "id": "training_group",
#   "users": [
#     "trainee1"
#   ]}

## Uncomment this to delete the samples
# delete_group = admin_client.studies.update_groups(study=study, action='REMOVE', data=group_data)
# delete_group.print_results(metadata=False, title='Info about groups')

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(delete_samples.get_results())<jupyter_output><empty_output><jupyter_text>3. Check users and groups in the StudyWe can always check the current status of the `@members` group i.e check which users has access to the study, as well that of any other group defined for the study. Hands-on exercise: Run the next cell after adding users `trainee1` and `trainee2`. Then remove them from the group `@members` (code cell below) and run it again. - Do you notice any difference in the output? Independently of the group **ALL** the users always belong to the group `@members`.<jupyter_code>## Check Study groups
groups = admin_client.studies.groups(study=study)
groups.print_results(fields='', metadata=False, title='Groups in Study {}'.format(study))

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(groups.get_results())<jupyter_output>Groups in Study demo@training:admin
----------------------------------------
#id	userIds
@members	trainee1,demo,imedina,pfurio,demo-admin,trainee2,llopez,jcoll,wspooner
@admins	imedina,pfurio,demo-admin,llopez,jcoll,wspooner
@training_group	trainee1<jupyter_text>Asign Samples to Individuals------- OpenCGA Catalog allows you to create samples entities. Then, you might need to create an individual entity associated with the sample.**[Important Note:]** When VCFs are ingested into OpenCGA, the pipeline automatically recognises the header of the VCF. If the name of the sample is present, it automatically creates the samples contained in the VCFs.  1. Create samplesFor this section, it is highly recommended to visit our documentation about the `Sample data model` at [**Sample Data Model**](https://app.gitbook.com/@opencb/s/opencga/manual/data-models/sample).First, let's create 3 samples: `sample1`, `sample2-germinal` and `sample2-somatic`. The most straight-forward way to create samples is using a loop, as showed in the next cell:<jupyter_code>## Create 3 samples taking into account if they're somatic or germinal
sample_ids = ['sample1', 'sample2-germinal', 'sample2-somatic']

for sample in sample_ids:
    if 'somatic' in sample:
        somatic=True
    else:
        somatic=False
    sample_data = {
      "id": sample,
      "description": "germinal sample",
      "processing": {
        "preparationMethod": "Illumina",
        "extractionMethod": "Parafin Embedded"
      },
      "collection": {
        "tissue": "skin",
        "organ": "skin",
        "method": "biopsy"
      },
      "somatic": somatic
    }

    admin_client.samples.create(study=study, data=sample_data)
    print('The sample: {} has been created successfully'.format(sample))<jupyter_output>The sample: sample1 has been created successfully
The sample: sample2-germinal has been created successfully
The sample: sample2-somatic has been created successfully<jupyter_text>- Check information about the created SamplesWe can use the `samples.search()` function to look for information about the recently created samples.<jupyter_code>## Search Samples
sample_ids = ['sample1', 'sample2-germinal', 'sample2-somatic']

for sample in sample_ids:
    samples = admin_client.samples.search(study=study, id=sample)
    samples.print_results(fields='id,collection,somatic', metadata=False, title='Info about sample {}'.format(sample))
    print('\n')

## Uncomment next line to display an interactive JSON viewer of the REST response
samples = admin_client.samples.search(study=study, id=sample_ids)
# JSON(samples.get_results())<jupyter_output><empty_output><jupyter_text>- Remove Samples:__[NOTE]:__ Be aware that if you delete the samples at this stage, you won't be able to associate the `samples` to the correspondent `individuals` in the next subsection.<jupyter_code>## Uncomment this to delete the samples
# delete_samples = admin_client.samples.delete(study=study, samples=['sample1,sample2-germinal,sample2-somatic'])

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(delete_samples.get_results())<jupyter_output><empty_output><jupyter_text>2. Create individuals Now let's create a couple of `individuals`. For this section, it is highly recommended to visit our documentation about the individual data model at [**Individual Data Model**](https://app.gitbook.com/@opencb/s/opencga/manual/data-models/individual).<jupyter_code>## Create 2 individuals 
individual_ids = ['individual1', 'individual2']

for individual in individual_ids:
    individual_data= {
  "id": individual,
  "name": individual,
  "location": {
    "postalCode": "CB1",
  },
  "sex": "MALE",
  "ethnicity": "black",
  "parentalConsanguinity":'true',
  "population": {
    "name": "spanish"
  },
  "dateOfBirth": "19900587",
  "karyotypicSex": "XY",
  "lifeStatus": "ALIVE"}
    admin_client.individuals.create(study=study, data=individual_data)
    print('The individual {} has been created succesfully'.format(individual))<jupyter_output><empty_output><jupyter_text>- Check information about the created individuals<jupyter_code>## Print information about the individuals
individual_ids = ['individual1', 'individual2']
for individual in individual_ids:
    individuals = admin_client.individuals.search(study=study, id=individual)
    individuals.print_results(fields='id,name,sex,ethnicity,lifeStatus,population,dateOfBirth', metadata=False, title='Info about individual {}'.format(individual))
    print('\n')
    
## Uncomment next line to display an interactive JSON viewer of the REST response
individuals = admin_client.individuals.search(study=study, id=individual_ids)
# JSON(individuals.get_results())<jupyter_output><empty_output><jupyter_text>3. Associate samples with individualsRemember that OpenCGA `individual` data model allows to have multiple samples assigned to the same individual. A typical use case can be found in cancer genetic screen, where usually two samples are taken from the same individual: one from the tumour (somatic sample), and one germinal.Bearing that in mind, we can create some individual entities in the database. In this case, let's assume that `sample1` corresponds to `individual1`, whilst `sample2-germinal` and `sample2-somatic` correspond to `individual2`.<jupyter_code>## Define a dictionary with the samples for each individual
individual_ids = ['individual1', 'individual2']
sample_dict={'individual1':['sample1'], 'individual2':['sample2-germinal', 'sample2-somatic']}

## Asign Samples to individuals
for individual in individual_ids:
    samples_associated = sample_dict[individual]
    admin_client.individuals.update(study=study, individuals=individual, data={'samples':samples_associated})
    print('The individual {} has been updated successfully!'.format(individual))<jupyter_output><empty_output><jupyter_text>- Check Samples associated to the Individuals<jupyter_code>## Print information about the individuals with their correspondent samples
individual_ids = ['individual1', 'individual2']
for individual in individual_ids:
    individuals = admin_client.individuals.search(study=study, id=individual)
    individuals.print_results(fields='id,samples.id,samples.somatic,samples.collection.tissue,samples.collection.method', metadata=False, title='Info about individual {} with the samples'.format(individual))
    print('\n')

## Uncomment next line to display an interactive JSON viewer of the REST response
individuals = admin_client.individuals.search(study=study, id=individual_ids, include='samples')
# JSON(individuals.get_results())<jupyter_output><empty_output><jupyter_text>- Delete Individuals<jupyter_code>## Uncomment this to delete the individuals
delete_individuals = admin_client.individuals.delete(study=study, individuals='individual1,individual2')

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(delete_individuals.get_results())<jupyter_output><empty_output><jupyter_text>Define phenotypes or disorders for Individuals------OpenCGA defines a set of rich data models, that can be useful for storing any kind of annotation. The individual entity can be associated with phenotypes and diseases ontologies. We support many ontology sources. Individuals that can be easily queried by its associated phenotypes and diseases once they are annotated.For this section, it is highly recommended to visit our documentation about the individual data model at [**Individual Data Model**](https://app.gitbook.com/@opencb/s/opencga/manual/data-models/individual).  1. Update already existing individuals with phenotypes - Create new `individuals`Lets create a couple of new individuals for this part of the notebook.<jupyter_code>## Create 2 new individuals
individual_ids = ['individual_A', 'individual_B']               
for individual in individual_ids:
    individual_data= {
  "id": individual,
  "name": individual,
  "location": {
    "postalCode": "47400A",
  },
  "sex": "FEMALE",
  "ethnicity": "black",
  "parentalConsanguinity":'false',
  "population": {
    "name": "british"
  },
  "dateOfBirth": "19900587",
  "karyotypicSex": "XX",
  "lifeStatus": "ALIVE"}
    admin_client.individuals.create(study=study, data=individual_data)
    print('The individual {} has been created succesfully'.format(individual))<jupyter_output><empty_output><jupyter_text>- Assign PhenotypesNow we can assign them the desired phenotypes by using `individuals.update()`. Remember that one individual might have more than one phenotype (`individual_B` in the next example).<jupyter_code>## Define a dictionary with the phenotyoes for each individual
individual_ids = ['individual_A', 'individual_B']               
phenotype_dict = {
    'individual_A':[{'id':'HP:0001626',
                    'name':'phenotype_A',
                    'source':'HPO',
                    'ageOfOnset':'15',
                    'status':'OBSERVED'}], 
    'individual_B':[{'id':'HP:0000119',
                     'name':'phenotype_B',
                     'source':'HPO',
                     'status':'OBSERVED'}, 
                    {'id':'HP:0000478',
                     'name':'phenotype_C',
                     'source':'HPO',
                     'status': 'OBSERVED'}]
                    }


## Assign Phenotypes to individuals
for individual in individual_ids:
    phenotypes_associated = phenotype_dict[individual]
    admin_client.individuals.update(study=study, individuals=individual, data={'phenotypes':phenotypes_associated})
    print('The individual {} has been updated successfully!'.format(individual))<jupyter_output><empty_output><jupyter_text>- Check Phenotypes of the Individuals<jupyter_code>## Print information about the individuals with their correspondent samples
individual_ids = ['individual_A', 'individual_B']               
for individual in individual_ids:
    individuals = admin_client.individuals.search(study=study, id=individual)
    individuals.print_results(fields='id,phenotypes.id,phenotypes.name,phenotypes.source,phenotypes.status', metadata=False, title='Info about the phenotypes of individual: {}'.format(individual))
    print('\n')

## Uncomment next line to display an interactive JSON viewer of the REST response
individuals = admin_client.individuals.search(study=study, id=individual_ids, include='phenotypes')
# JSON(individuals.get_results())<jupyter_output>Info about the phenotypes of individual: individual_A
----------------------------------------------------------
#id	phenotypes.id	phenotypes.name	phenotypes.source	phenotypes.status
individual_A	HP:0001626	phenotype_A	HPO	OBSERVED


Info about the phenotypes of individual: individual_B
----------------------------------------------------------
#id	phenotypes.id	phenotypes.name	phenotypes.source	phenotypes.status
individual_B	HP:0000119,HP:0000478	phenotype_B,phenotype_C	HPO	OBSERVED<jupyter_text>- Assign DisordersSimilarly to phenotypes, we can also update the disorders for an Individual.<jupyter_code>## Define a dictionary with the Disorders for each Individual
individual_ids = ['individual_A', 'individual_B']               
disorders_dict = {
    'individual_A':[{'id':'OMIM:300600',
                    'name':'disorder_A',
                    'source':'OMIM',
                    'description':'Aland Island Eye Disease'}], 
    'individual_B':[{'id':'OMIM:203450',
                     'name':'disorder_B',
                     'source':'OMIM',
                     'description':'Alexander Disease'}, 
                    {'id':'OMIM:104300',
                     'name':'disorder_C',
                     'source':'OMIM',
                     'description':'Alzheimer Disease'}]
                    }

## Assign Disorders to Individuals
for individual in individual_ids:
    disorders_associated = disorders_dict[individual]
    admin_client.individuals.update(study=study, individuals=individual, data={'disorders':disorders_associated})
    print('The individual {} has been updated successfully!'.format(individual))<jupyter_output><empty_output><jupyter_text>- Check Disorders of the Individuals<jupyter_code>## Print information about the individuals with their correspondent samples
individual_ids = ['individual_A', 'individual_B']               
for individual in individual_ids:
    individuals = admin_client.individuals.search(study=study, id=individual)
    individuals.print_results(fields='id,disorders.id,disorders.name,disorders.source,disorders.description', metadata=False, title='Info about the disorders of individual: {}'.format(individual))
    print('\n')

## Uncomment next line to display an interactive JSON viewer of the REST response
individuals = admin_client.individuals.search(study=study, id=individual_ids, include='disorders')
# JSON(individuals.get_results())<jupyter_output><empty_output><jupyter_text>- Delete Individuals<jupyter_code>## Uncomment this to delete the individuals
delete_individuals = admin_client.individuals.delete(study=study, individuals=individual_ids)

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(delete_individuals.get_results())<jupyter_output><empty_output><jupyter_text>2. Create individual with associated phenotypes and disordersIf we have phenotype and disorder information about one individual at the time of creating the entity in OpenCGA Catalog, we cal always create the individual with all the associated information directly.<jupyter_code>## Construct an individual data model
individual_data = {
  "id": "Individual_with_data",
  "name": "Jonh",
  "location": {
    "city": "New York",
    "country": "USA"
  },
  "sex": "MALE",
  "ethnicity": "american",
  "dateOfBirth": "18800722",
  "karyotypicSex": "XY",
  "lifeStatus": "ALIVE",
  "phenotypes": [{'id':'HP:0001626',
                  'name':'phenotype_A',
                  'source':'HPO',
                  'ageOfOnset':'15',
                  'status':'OBSERVED'}],
  "disorders": [{'id':'OMIM:104300',
                 'name':'disorder_C',
                 'source':'OMIM',
                 'description':'Alzheimer Disease',
                 'evidences':[{
                      'id':'HP:0001626',
                      'name':'phenotype_A',
                      'source':'HPO',
                  'ageOfOnset':'15',
                  'status':'OBSERVED'}]}]
                }

## Create the individual
Individual_with_data = admin_client.individuals.create(study=study, data=individual_data)
print('The individual has been successfully created')<jupyter_output><empty_output><jupyter_text>- Hands-on Exercise: Check the individual data modelTun the next cell to display the data from the created individual using the interactive JSON viewer. Now compare the data with the individual data model at [**Individual Data Model**](https://app.gitbook.com/@opencb/s/opencga/manual/data-models/individual). Can you see the parallelism?<jupyter_code>## Display an interactive JSON viewer of the REST response
individuals = admin_client.individuals.search(study=study, id='Individual_with_data')
JSON(individuals.get_results())<jupyter_output><empty_output><jupyter_text>- Delete the individual<jupyter_code>## Uncomment this to delete the individuals
delete_individuals = admin_client.individuals.delete(study=study, individuals='Individual_with_data')

## Uncomment next line to display an interactive JSON viewer of the REST response
# JSON(delete_individuals.get_results())<jupyter_output><empty_output><jupyter_text>Define, add, edit and remove variable sets----------One incredibly practical utility of OpenCGA is the possibility of defining variable sets (i.e: groups of variables) for a specific entity or group of entities (Samples, Individuals, Cohorts, etc.)For more information, and before running this part of the notebook, refer to the section of [Adding custom metadata](https://app.gitbook.com/@opencb/s/opencga/manual/data-management/adding-custom-metadata) in our OpenCGA GitBook Documentation.  1. Define the Variable SetIn this case, we will create a Variable set for the *annotable* entity `Sample`. According to our documentation, we consider an entry to be Annotable if the entry can have Annotation Sets. At this stage, only **File, Sample, Individual, Cohort and Family** are Annotable.<jupyter_code>## Define the variable set
sample_variable_set = {
  "id": "sample_metadata",  
  "unique": "true",
  "confidential": "false",
  "description": "Sample origin",
  "entities": [
    "SAMPLE"
  ],
  "variables": [
    {
      "id": "cell_line",
      "name": "Cell line",
      "category": "string",
      "type": "TEXT",
      "required": "false",
      "multiValue": "false",
      "allowedValues": [],
      "rank": 2,
      "dependsOn": "",
      "description": "Sample cell line",
      "attributes": {}
    },
      {
      "id": "cell_type",
      "name": "Cell type",
      "category": "string",
      "type": "TEXT",
      "required": "false",
      "multiValue": "false",
      "allowedValues": [],
      "rank": 3,
      "dependsOn": "",
      "description": "Sample cell type",
      "attributes": {}
    }]
}

## Add the variable set to the Sample data model of the study
add_variable_set = admin_client.studies.update_variable_sets(study=study, action='ADD', data=sample_variable_set)
## Search for the variable set
variable_sets = admin_client.studies.variable_sets(study=study)
variable_sets.print_results(fields='id,name,unique,description', metadata=False, title='List of Variable Sets in study: {}'.format(study))<jupyter_output>List of Variable Sets in study: demo@training:admin
--------------------------------------------------------
#id	name	unique	description
opencga_file_variant_stats	opencga_file_variant_stats	True	OpenCGA file variant stats
opencga_alignment_stats	opencga_alignment_stats	True	OpenCGA alignment stats
opencga_sample_qc	opencga_sample_qc	True	OpenCGA variable set for sample quality control (QC)
opencga_cohort_variant_stats	opencga_cohort_variant_stats	True	OpenCGA cohort variant stats
sample_metadata	sample_metadata	True	Sample origin<jupyter_text>- Remove Variable Set<jupyter_code>## Remove the variable set to the Sample data model of the study
#remove_variable_set = admin_client.studies.update_variable_sets(study=study, action='REMOVE', data=sample_variable_set)<jupyter_output><empty_output><jupyter_text>2. Annotate the Variable Set: Annotation Set<jupyter_code>## Define the annotation set
sample_annotation_set = [{
    "id": "annotation_set_id",
    "variableSetId": "sample_metadata",
    "annotations": {
    "cell_line": "umbilical cord",
    "cell_type": "multipotent progenitor",
  }
}]

## Annotate the variable set
admin_client.samples.update(samples='sample1', data={'annotationSets':sample_annotation_set})
samples = admin_client.samples.search(study=study)

## Uncomment next line to display an interactive JSON viewer of the REST response
JSON(samples.get_results())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11134543776979602, 'fraction_numerical': 0.007315385375819686, 'mean_word_length': 4.806007269944518, 'pattern_counts': {'":': 87, '<': 112, '<?xml version=': 0, '>': 112, 'https://': 15, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '29', 'dir': 'jupyter-structured-clean-dedup', 'id': '320734', 'license': 'Apache-2.0', 'n_tokens_mistral': 9203, 'n_tokens_neox': 8466, 'n_words': 2709, 'path': 'opencga-client/src/main/python/notebooks/admin-training/pyopencga_admin.ipynb', 'repo_name': 'lauralopezreal/opencga-laura', 'size': '30347'}