<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "modern-charge",
   "metadata": {},
   "source": [
    "# *pyopencga* Catalog: Clinical Data and other Metadata. \n",
    "\n",
    "------\n",
    "\n",
    "## Overview\n",
    "\n",
    "This notebook is intended to provide guidance for querying an OpenCGA server through *pyopencga* to explore studies which the user has access to, Clinical data provided in the study (Samples, Individuals Genotypes etc.) and other types of metadata, like permissions.\n",
    "\n",
    "A good first step when start working with OpenCGA is to retrieve information about our user, which projects and studies are we allowed to see.<br>\n",
    "It is also recommended to get a taste of the clinical data we are encountering in the study: How many samples and individuals does the study have? Is there any defined cohorts? Can we get some statistics about the genotypes of the samples in the Sudy?\n",
    "\n",
    "For guidance on how to loggin and get started with *opencga* you can refer to : [001-pyopencga_first_steps.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)\n",
    " \n",
    " **[NOTE]** The server methods used by *pyopencga* client are defined in the following swagger URL:\n",
    "- https://ws.opencb.org/opencga-prod/webservices/   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proved-cassette",
   "metadata": {},
   "source": [
    "## 1. Setup the Client and Login into *pyopencga* \n",
    "\n",
    "**Configuration and Credentials** \n",
    "\n",
    "Let's assume we already have *pyopencga* installed in our python setup (all the steps described on [001-pyopencga_first_steps.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)).\n",
    "\n",
    "You need to provide **at least** a host server URL in the standard configuration format for OpenCGA as a python dictionary or in a json file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "hollywood-lewis",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Config information:\n",
      " {'rest': {'host': 'https://ws.opencb.org/opencga-prod'}}\n"
     ]
    }
   ],
   "source": [
    "from pyopencga.opencga_config import ClientConfiguration # import configuration module\n",
    "from pyopencga.opencga_client import OpencgaClient # import client module\n",
    "from pprint import pprint\n",
    "import json\n",
    "\n",
    "####################################\n",
    "## Configuration parameters  #######\n",
    "####################################\n",
    "# OpenCGA host\n",
    "host = 'https://ws.opencb.org/opencga-prod'\n",
    "\n",
    "# User credentials\n",
    "user = 'demouser'\n",
    "passwd = '<PASSWORD>' ## you can skip this, see below.\n",
    "study = 'demo@family:platinum'\n",
    "####################################\n",
    "\n",
    "# Creating ClientConfiguration dict\n",
    "config_dict = {'rest': {\n",
    "                       'host': host \n",
    "                    }\n",
    "               }\n",
    "print('Config information:\\n',config_dict)\n",
    "\n",
    "# Pass the config_dict dictionary to the ClientConfiguration method\n",
    "config = ClientConfiguration(config_dict)\n",
    "\n",
    "# Create the client\n",
    "oc = OpencgaClient(config)\n",
    "\n",
    "# Pass the credentials to the client\n",
    "# (here we put only the user in order to be asked for the password interactively)\n",
    "# oc.login(user)\n",
    "\n",
    "# or you can pass the user and passwd\n",
    "oc.login(user, passwd)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "devoted-rental",
   "metadata": {},
   "source": [
    "Once we have defined a variable with the client configuration and credentials, we can access to all the methods defined for the client. These methods implement calls to query different data models in *OpenCGA*. \n",
    "\n",
    "Over the user case addressed in this notebook we will be performing queries to the **users, projects, studies, samples, individuals and cohorts**<br> *OpenCGA* data models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "collective-reverse",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Define variables to query different data models through the web services\n",
    "\n",
    "user_client = oc.users\n",
    "project_client = oc.projects\n",
    "study_client = oc.studies\n",
    "sample_client = oc.samples\n",
    "individual_client = oc.individuals\n",
    "file_client = oc.files\n",
    "cohort_client = oc.cohorts\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "european-gabriel",
   "metadata": {},
   "source": [
    "# 2. Use Cases \n",
    "\n",
    "In this seciton we are going to show how to work with some of the most common scenarios.<br>\n",
    "The user-cases addresed here constute a high-level introduction aimed to provide a basis for the user to make their own explorations. This example can be adapted to each individual user-case.\n",
    "\n"<commit_msg>client: changes in catalog notebook<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "modern-charge",
   "metadata": {},
   "source": [
    "# Overview\n",
    "------\n",
    "This notebook is intended to provide guidance for querying an OpenCGA server through *pyopencga* to explore studies which the user has access to, Clinical data provided in the study (Samples, Individuals Genotypes etc.) and other types of metadata, like permissions.\n",
    "\n",
    "A good first step when start working with OpenCGA is to retrieve information about our user, which projects and studies are we allowed to see.<br>\n",
    "It is also recommended to get a taste of the clinical data we are encountering in the study: How many samples and individuals does the study have? Is there any defined cohorts? Can we get some statistics about the genotypes of the samples in the Sudy?\n",
    "\n",
    "For guidance on how to loggin and get started with *opencga* you can refer to : [001-pyopencga_first_steps.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)\n",
    " \n",
    " **[NOTE]** The server methods used by *pyopencga* client are defined in the following swagger URL:\n",
    "- https://ws.opencb.org/opencga-prod/webservices/   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "proved-cassette",
   "metadata": {},
   "source": [
    "## Setup the Client and Login into *pyopencga* \n",
    "\n",
    "**Configuration and Credentials** \n",
    "\n",
    "Let's assume we already have *pyopencga* installed in our python setup (all the steps described on [001-pyopencga_first_steps.ipynb](https://github.com/opencb/opencga/tree/develop/opencga-client/src/main/python/notebooks/user-training)).\n",
    "\n",
    "You need to provide **at least** a host server URL in the standard configuration format for OpenCGA as a python dictionary or in a json file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "hollywood-lewis",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Config information:\n",
      " {'rest': {'host': 'https://ws.opencb.org/opencga-prod'}}\n"
     ]
    }
   ],
   "source": [
    "from pyopencga.opencga_config import ClientConfiguration # import configuration module\n",
    "from pyopencga.opencga_client import OpencgaClient # import client module\n",
    "from pprint import pprint\n",
    "import json\n",
    "\n",
    "####################################\n",
    "## Configuration parameters  #######\n",
    "####################################\n",
    "# OpenCGA host\n",
    "host = 'https://ws.opencb.org/opencga-prod'\n",
    "\n",
    "# User credentials\n",
    "user = 'demouser'\n",
    "passwd = '<PASSWORD>' ## you can skip this, see below.\n",
    "study = 'demo@family:platinum'\n",
    "####################################\n",
    "\n",
    "# Creating ClientConfiguration dict\n",
    "config_dict = {'rest': {\n",
    "                       'host': host \n",
    "                    }\n",
    "               }\n",
    "print('Config information:\\n',config_dict)\n",
    "\n",
    "# Pass the config_dict dictionary to the ClientConfiguration method\n",
    "config = ClientConfiguration(config_dict)\n",
    "\n",
    "# Create the client\n",
    "oc = OpencgaClient(config)\n",
    "\n",
    "# Pass the credentials to the client\n",
    "# (here we put only the user in order to be asked for the password interactively)\n",
    "# oc.login(user)\n",
    "\n",
    "# or you can pass the user and passwd\n",
    "oc.login(user, passwd)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "devoted-rental",
   "metadata": {},
   "source": [
    "Once we have defined a variable with the client configuration and credentials, we can access to all the methods defined for the client. These methods implement calls to query different data models in *OpenCGA*. \n",
    "\n",
    "Over the user case addressed in this notebook we will be performing queries to the **users, projects, studies, samples, individuals and cohorts**<br> *OpenCGA* data models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "collective-reverse",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Define variables to query different data models through the web services\n",
    "\n",
    "user_client = oc.users\n",
    "project_client = oc.projects\n",
    "study_client = oc.studies\n",
    "sample_client = oc.samples\n",
    "individual_client = oc.individuals\n",
    "file_client = oc.files\n",
    "cohort_client = oc.cohorts\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "european-gabriel",
   "metadata": {},
   "source": [
    "##  Use Cases \n",
    "\n",
    "In this seciton we are going to show how to work with some of the most common scenarios.<br>\n",
    "The user-cases addresed here constute a high-level introduction aimed to provide a basis for the user to make their own explorations. This example can be adapted to each individual user-case.\n"
{'fraction_non_alphanumeric': 0.16769825918762088, 'fraction_numerical': 0.0019342359767891683, 'mean_word_length': 3.02843786521231, 'pattern_counts': {'":': 64, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6819257', 'n_tokens_mistral': 3312, 'n_tokens_neox': 3230, 'n_words': 1039}