<jupyter_start><jupyter_text>Exploring HTR3A protein target activity data from Drug Target Commons  Scope of the tutorial*   Input data from Excape dataset for HTR3A protein target*   AMPL will be used to accomplish the following steps:   *   Standardize SMILES string  *   Clean the data (look for duplicates, average the assay data, cluster the compounds etc.)  *   Curate the data  *   Carry out some Exploratory Data Analysis (Chemical space exploration; PCA etc.)   *   Save the final curated dataset for modeling Diseases associated with HTR3A include Irritable Bowel Syndrome and Motion Sickness.  Time on COLAB-Pro ( ~ 3 minutes)  Protein target (HTR3A) information The HTR3A target binding data was downloaded from DTC database, https://drugtargetcommons.fimm.fi.Please refer to the following publication,https://pubmed.ncbi.nlm.nih.gov/29276046/, for details about the database  Additional information about HTR3A gene:**Gene location:** Chromosome 11**Exon count:** 10mRNA and protein information for its three transcripts:*   NM_000869.6 → NP_000860.3 *   NM_001161772.3 → NP_001155244.1*   NM_213621.4 → NP_998786.3   Before you begin, make sure you close all other COLAB notebooks.   Change Runtime settingsIf you have access to COLAB-Pro (commercial/not-free), please change your runtime settings to use GPU and high-memory,```Runtime --> Change Runtime Type --> GPU with high-RAM```If you are not a paid COLAB-Pro customer, you can still choose GPU, with standard-RAM.<jupyter_code>!date # starting time<jupyter_output>Fri Sep 17 11:54:43 UTC 2021<jupyter_text>Install AMPL<jupyter_code>! pip install rdkit-pypi
! pip install --pre deepchem

import deepchem
# print(deepchem.__version__)
! pip install umap
! pip install llvmlite==0.34.0  --ignore-installed
! pip install umap-learn
! pip install molvs
! pip install bravado
import deepchem as dc

# get the Install AMPL_GPU_test.sh
!wget 'https://raw.githubusercontent.com/ATOMScience-org/AMPL/master/atomsci/ddm/examples/tutorials/config/install_AMPL_GPU_test.sh'

# run the script to install AMPL
! chmod u+x install_AMPL_GPU_test.sh
! ./install_AMPL_GPU_test.sh<jupyter_output><empty_output><jupyter_text>Exploring target activity data from selected database Data source = ExcapeDB Target activity for the receptor HTR3A<jupyter_code># We temporarily disable warnings for demonstration.
# FutureWarnings and DeprecationWarnings are present from some of the AMPL 
# dependency modules.
import warnings
warnings.filterwarnings('ignore')
import json
import os
import requests
#
# Import AMPL libraries
# 
import atomsci.ddm.utils.data_curation_functions as dcf
import atomsci.ddm.utils.curate_data as curate_data
import atomsci.ddm.pipeline.diversity_plots as dp
import atomsci.ddm.pipeline.chem_diversity as cd
# Additional python libraries
import pandas as pd
import numpy as np
import getpass,os<jupyter_output><empty_output><jupyter_text>Select a target to work with  (e.g. PDE2A, KCNH2, SCNA5)<jupyter_code>target_name='HTR3A'<jupyter_output><empty_output><jupyter_text>Define data locations<jupyter_code>ofile=target_name+'_dtc_smiles.csv'<jupyter_output><empty_output><jupyter_text>Note the file `HTR3A_dtc_smiles.csv` is the output fromhttps://github.com/ATOMScience-org/AMPL/blob/master/atomsci/ddm/examples/tutorials/03_Explore_Data_DTC.ipynb notebook<jupyter_code>import io
url = 'https://raw.githubusercontent.com/ATOMScience-org/AMPL/master/atomsci/ddm/examples/tutorials/datasets/HTR3A_dtc_smiles.csv'
download = requests.get(url).content
# Reading the downloaded content and turning it into a pandas dataframe
save_smiles_df = pd.read_csv(io.StringIO(download.decode('utf-8')), sep=',', header=0 )
# save_smiles_df.rename( columns={ "PIC50" : "PXC50" }, inplace=True)<jupyter_output><empty_output><jupyter_text>Check for assay replicates with high measurement variation and flag them<jupyter_code>import os

# From our dataframe, we are working with the PIC50 column 
data=save_smiles_df

# Here we are defining our response column, which will be PIC50
column = 'PIC50'

# tolerance: Percentage of individual respsonse values allowed to different from 
# the average to be included in averaging
tolerance = 10

# list_bad_duplicates: Print structures with bad duplicates
list_bad_duplicates = 'Yes'

# max_std: Maximum allowed standard deviation for computed average response value
# NOTE: In this example, we set this value very high to disable this feature
max_std = 1

# compound_id: Compound ID column
compound_id = 'standard_inchi_key'

# smiles_col: SMILES column
smiles_col = 'rdkit_smiles'

# Here we are creating a new dataframe, called check_df
check_df = curate_data.average_and_remove_duplicates(column, tolerance, 
                                                       list_bad_duplicates, 
                                                       data, max_std, 
                                                       compound_id=compound_id, 
                                                       smiles_col=smiles_col)<jupyter_output>Bad duplicates removed from dataset
Dataframe size (197, 52)
List of 'bad' duplicates removed
              standard_inchi_key     PIC50  ...   Perc_Var  VALUE_NUM_std
172  FZGWDCDUMFWUEO-UHFFFAOYSA-N  8.159894  ...  11.975455       1.234152
173  FZGWDCDUMFWUEO-UHFFFAOYSA-N  6.414539  ...  11.975455       1.234152
67   KQKPFRSPSRPDEB-UHFFFAOYSA-N  5.000000  ...  23.262963       2.143606
68   KQKPFRSPSRPDEB-UHFFFAOYSA-N  8.031517  ...  23.262963       2.143606

[4 rows x 5 columns]

Dataset de-duplicated
Dataframe size (170, 52)
New column created with averaged values:  VALUE_NUM_mean<jupyter_text>Examine the rejected compounds and examine them<jupyter_code>old_compound_id='rdkit_smiles'
new_compound_id='rdkit_smiles'

# Takes all the compounds that aren't part of the curated data frame and prints them
reject=data[~data[old_compound_id].isin(check_df[new_compound_id])]
reject
# Prints number of unique rejected compound names and rows
print("Number of discarded compounds",reject[compound_id].nunique())

# Apecifies directory for csv file to send rejected data to, saving it as ofile 
ofile = target_name+'.reject_check'

# Sends the rejected compound dataframe to a csv file with the path (ofile) specified above
reject.to_csv(ofile,index=False)
# Prints all the results of the unique assay tests run on this target
save_smiles_df['assay_description'].unique()
# Collects all the non-rejected data into a new frame called data
data=save_smiles_df[~save_smiles_df[compound_id].isin(reject[compound_id])]

# Prints the rows & columns (in that order) of the dataset from above (i.e. all the matching data)
# Shape is an array that can check how large the dataframe is 
# It will print out (x, y), the x is the rows, or lines, and y is the number of columns 
print(data.shape)
data[['standard_inchi_key','rdkit_smiles','standard_relation']].head(5)
#specifies the column we want (response value) and also tells it to include the duplicates
column='PIC50'; #'standard_value'
list_bad_duplicates='Yes'

# Actually compiles all the data with the specifies column and tells what columns to include 
# (inchi_key, smiles, and relation)
temp_df=curate_data.aggregate_assay_data(data, 
                                         value_col=column, 
                                         output_value_col=None,
                                         label_actives=True,
                                         active_thresh=6,
                                         id_col='standard_inchi_key', 
                                         smiles_col='rdkit_smiles', 
                                         relation_col='standard_relation')

#Removes all duplicates from the data frame and puts it in the final, curated dataframe
curated_df = temp_df[~temp_df.isin([np.inf]).any(1)]<jupyter_output>0 entries in input table are missing SMILES strings
170 unique SMILES strings are reduced to 170 unique base SMILES strings<jupyter_text>Save data to a file<jupyter_code># prints the number of rows in the before and after curation 
print("before",data.shape[0],"after",curated_df.shape[0])

#prints the first few rows of the "after" (after curation) dataframes
curated_df.head()
# Prints number of unique resjected compound names/rows
print("Number of discarded compounds",reject[compound_id].nunique())

# Apecifies directory for csv file to send rejected data to, saving it as ofile 
ofile = target_name+'.reject_check'

# Sends the rejected compound dataframe to a csv file with the path (ofile) specified above
reject.to_csv(ofile,index=False)
# specifies file path to save the curated data to
ofile = target_name+'_dtc_smiles_curated.csv'

# sends data from the curated_df to csv with specified path above
curated_df.to_csv(ofile,index=False)
# load graphics libraries matplotlib and seaborn (sns)
import matplotlib.pyplot as plt
import seaborn as sns

# plt.subplots is a function that creates a full grid of subplots in a single array, and returning 
# them in a NumPy array
# A NumPy array is a grid of values, all of the same type, and is indexed by a tubule of nonnegative integers 
# We are saving this subplots we're creating as fig, ax variables
fig, ax = plt.subplots()

# compiles the data frames before and after all the curation with the specified column value (response values)
# If we remember or look up further in the notebook, we created two dataframes: data and curated_df 
# Our data dataframe consists of ofile (target_name+'_excape_curated.csv.') and we will be using the PIC50 column of our dataframe
# This will be our before variable
before=save_smiles_df["PIC50"]

# Our after variable is comprised of our curated_df dataframe 
# Our curated_df was configured earlier to discard any out of range values
after=curated_df["PIC50"]

# plots the values of the response value column of both the before and after dataframes from above
# sns.distplot is a Seaborn distplot, which allows you to show a histogram with a line on it
# A distplot plots a univariate distribution of observations 
# Below, we are separating our data set into a before section and an after section 
# To break down the sns.distplot functions: 
# bins=range(0, 12, 1) is used to bin our range of values, meaning to divide the entire range of values into a series of intervals
# ax=ax 
# kde is a kernel density estimate plot that represents the data using a continuous probability density curve
# kde=False will remove the density line on the histogram
# The label= function is used to label different sections of our plot 
sns.distplot(before, bins=range(0, 12, 1), ax=ax, kde=False, label='before')
sns.distplot(after, bins=range(0, 12, 1), ax=ax, kde=False, label='after')

#configures plot by adding a legend and setting the x axis limits for readability
# plt.legend() adds the legend to the plot
plt.legend()

# ax.set_xlim is setting the x-axis limits of the graph, setting it to be from (0,12)
ax.set_xlim([0, 12])

!date # ending time<jupyter_output>Fri Sep 17 11:56:12 UTC 2021
{'fraction_non_alphanumeric': 0.0705925052173124, 'fraction_numerical': 0.02440794846202704, 'mean_word_length': 4.102777777777778, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 35, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 26, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '154784', 'license': 'MIT', 'n_tokens_mistral': 3580, 'n_tokens_neox': 3266, 'n_words': 1193, 'path': 'atomsci/ddm/examples/tutorials/04_Explore_Data_DTC_Curate.ipynb', 'repo_name': 'stewarthe6/AMPL-1', 'size': '11021'}