<jupyter_start><jupyter_text>**Computational Drug Discovery [Part 1] Download Bioactivity Data**<NAME>[*'Data Professor' YouTube channel*](http://youtube.com/dataprofessor)In this Jupyter notebook, we will be building a real-life **data science project** that you can include in your **data science portfolio**. Particularly, we will be building a machine learning model using the ChEMBL bioactivity data.---  **ChEMBL Database**The [*ChEMBL Database*](https://www.ebi.ac.uk/chembl/) is a database that contains curated bioactivity data of more than 2 million compounds. It is compiled from more than 76,000 documents, 1.2 million assays and the data spans 13,000 targets and 1,800 cells and 33,000 indications.[Data as of March 25, 2020; ChEMBL version 26].  **Installing libraries** Install the ChEMBL web service package so that we can retrieve bioactivity data from the ChEMBL Database.<jupyter_code>! pip install chembl_webresource_client<jupyter_output>Requirement already satisfied: chembl_webresource_client in /usr/local/lib/python3.7/dist-packages (0.10.7)
Requirement already satisfied: requests>=2.18.4 in /usr/local/lib/python3.7/dist-packages (from chembl_webresource_client) (2.23.0)
Requirement already satisfied: requests-cache~=0.7.0 in /usr/local/lib/python3.7/dist-packages (from chembl_webresource_client) (0.7.4)
Requirement already satisfied: easydict in /usr/local/lib/python3.7/dist-packages (from chembl_webresource_client) (1.9)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/dist-packages (from chembl_webresource_client) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.18.4->chembl_webresource_client) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.18.4->chembl_webresource_client) (2021.5.30)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dis[...]<jupyter_text>**Importing libraries**<jupyter_code># Import necessary libraries
import pandas as pd
from chembl_webresource_client.new_client import new_client<jupyter_output><empty_output><jupyter_text>**Search for Target protein**  **Target search for coronavirus**<jupyter_code># Target search for coronavirus
target = new_client.target
target_query = target.search('coronavirus')
targets = pd.DataFrame.from_dict(target_query)
targets<jupyter_output><empty_output><jupyter_text>**Select and retrieve bioactivity data for *SARS coronavirus 3C-like proteinase* (fifth entry)** We will assign the fifth entry (which corresponds to the target protein, *coronavirus 3C-like proteinase*) to the ***selected_target*** variable<jupyter_code>selected_target = targets.target_chembl_id[4]
selected_target<jupyter_output><empty_output><jupyter_text>Here, we will retrieve only bioactivity data for *coronavirus 3C-like proteinase* (CHEMBL3927) that are reported as IC$_{50}$ values in nM (nanomolar) unit.<jupyter_code>activity = new_client.activity
res = activity.filter(target_chembl_id=selected_target).filter(standard_type="IC50")
df = pd.DataFrame.from_dict(res)
df.head(3)
df.standard_type.unique()<jupyter_output><empty_output><jupyter_text>Finally we will save the resulting bioactivity data to a CSV file **bioactivity_data.csv**.<jupyter_code>df.to_csv('bioactivity_data.csv', index=False)<jupyter_output><empty_output><jupyter_text>**Copying files to Google Drive** Firstly, we need to mount the Google Drive into Colab so that we can have access to our Google adrive from within Colab.<jupyter_code>!pip install drive
from google.colab import drive
drive.mount('/content/gdrive/', force_remount=True)<jupyter_output>Mounted at /content/gdrive/<jupyter_text>Next, we create a **data** folder in our **Colab Notebooks** folder on Google Drive.<jupyter_code>! mkdir "/content/gdrive/My Drive/Colab Notebooks/data2"
! cp bioactivity_data.csv "/content/gdrive/My Drive/Colab Notebooks/data"
! ls -l "/content/gdrive/My Drive/Colab Notebooks/data"<jupyter_output>-rw------- 1 root root 70334 Aug 21 22:07 '/content/gdrive/My Drive/Colab Notebooks/data'<jupyter_text>Let's see the CSV files that we have so far.<jupyter_code>! ls<jupyter_output>bioactivity_data.csv  bioactivity_preprocessed_data.csv  gdrive  sample_data<jupyter_text>Taking a glimpse of the **bioactivity_data.csv** file that we've just created.<jupyter_code>! head bioactivity_data.csv<jupyter_output>activity_comment,activity_id,activity_properties,assay_chembl_id,assay_description,assay_type,assay_variant_accession,assay_variant_mutation,bao_endpoint,bao_format,bao_label,canonical_smiles,data_validity_comment,data_validity_description,document_chembl_id,document_journal,document_year,ligand_efficiency,molecule_chembl_id,molecule_pref_name,parent_molecule_chembl_id,pchembl_value,potential_duplicate,qudt_units,record_id,relation,src_id,standard_flag,standard_relation,standard_text_value,standard_type,standard_units,standard_upper_value,standard_value,target_chembl_id,target_organism,target_pref_name,target_tax_id,text_value,toid,type,units,uo_units,upper_value,value
,1480935,[],CHEMBL829584,In vitro inhibitory concentration against SARS coronavirus main protease (SARS CoV 3C-like protease),B,,,BAO_0000190,BAO_0000357,single protein format,Cc1noc(C)c1CN1C(=O)C(=O)c2cc(C#N)ccc21,,,CHEMBL1139624,Bioorg. Med. Chem. Lett.,2005,"{'bei': '18.28', 'le': '0.33', 'lle': '3.25', 'sei': '5.90'}[...]<jupyter_text>**Handling missing data**If any compounds has missing value for the **standard_value** column then drop it<jupyter_code>df2 = df[df.standard_value.notna()]
df2<jupyter_output><empty_output><jupyter_text>Apparently, for this dataset there is no missing data. But we can use the above code cell for bioactivity data of other target protein.  **Data pre-processing of the bioactivity data**  **Labeling compounds as either being active, inactive or intermediate**The bioactivity data is in the IC50 unit. Compounds having values of less than 1000 nM will be considered to be **active** while those greater than 10,000 nM will be considered to be **inactive**. As for those values in between 1,000 and 10,000 nM will be referred to as **intermediate**.<jupyter_code>bioactivity_class = []
for i in df2.standard_value:
  if float(i) >= 10000:
    bioactivity_class.append("inactive")
  elif float(i) <= 1000:
    bioactivity_class.append("active")
  else:
    bioactivity_class.append("intermediate")<jupyter_output><empty_output><jupyter_text>**Iterate the *molecule_chembl_id* to a list**<jupyter_code>molecule_chembl_id  = []
for i in df2.molecule_chembl_id:
  molecule_chembl_id.append(i)<jupyter_output><empty_output><jupyter_text>**Iterate *canonical_smiles* to a list**<jupyter_code>canonical_smiles = []
for i in df2.canonical_smiles:
  canonical_smiles.append(i)<jupyter_output><empty_output><jupyter_text>**Iterate *standard_value* to a list**<jupyter_code>standard_value = []
for i in df2.standard_value:
  standard_value.append(i)<jupyter_output><empty_output><jupyter_text>**Combine the 4 lists into a dataframe**<jupyter_code>data_tuples = list(zip(molecule_chembl_id, canonical_smiles, bioactivity_class, standard_value))
df3 = pd.DataFrame( data_tuples,  columns=['molecule_chembl_id', 'canonical_smiles', 'bioactivity_class', 'standard_value'])
df3<jupyter_output><empty_output><jupyter_text>**Alternative method**<jupyter_code>selection = ['molecule_chembl_id', 'canonical_smiles', 'standard_value']
df3 = df2[selection]
df3
pd.concat([df3,pd.Series(bioactivity_class)], axis=1)<jupyter_output><empty_output><jupyter_text>Saves dataframe to CSV file<jupyter_code>df3.to_csv('bioactivity_preprocessed_data.csv', index=False)
! ls -l<jupyter_output>total 92
-rw-r--r-- 1 root root 70334 Aug 21 22:07 bioactivity_data.csv
-rw-r--r-- 1 root root  9337 Aug 21 22:07 bioactivity_preprocessed_data.csv
drwx------ 5 root root  4096 Aug 21 22:07 gdrive
drwxr-xr-x 1 root root  4096 Aug 13 13:35 sample_data<jupyter_text>Let's copy to the Google Drive<jupyter_code>! cp bioactivity_preprocessed_data.csv "/content/gdrive/My Drive/Colab Notebooks/data"
! ls "/content/gdrive/My Drive/Colab Notebooks/data"<jupyter_output>'/content/gdrive/My Drive/Colab Notebooks/data'
{'fraction_non_alphanumeric': 0.11008393139520739, 'fraction_numerical': 0.03393747719255565, 'mean_word_length': 7.600418410041841, 'pattern_counts': {'":': 0, '<': 74, '<?xml version=': 0, '>': 80, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '506824', 'license': 'MIT', 'n_tokens_mistral': 2970, 'n_tokens_neox': 2717, 'n_words': 642, 'path': 'Drug Discovery Using Machine Learning and Data Analysis/CDD_ML_Part_1_bioactivity_data.ipynb', 'repo_name': 'edaaydinea/Drug-Discovery-Using-Machine-Learning-and-Data-Analysis', 'size': '8234'}