<jupyter_start><jupyter_text>PIPELINE FOR INTERACTIONS ANALYSIS OF 1UTS HIV-1 TAR STRUCTURE WITH ITS ACTIVE/INACTIVE LIGANDSOmit points 1-3 to repeat analysis on precalculated Structural Interactions Fingerprints (SIFs) on HIV-1 TAR active/inactive ligands dataset docked to 1UTS.  Otherwise create the dataset, perform molecular docking and run all the cells to repeat the entire analysis.  1. Prepare HIV-1 TAR active/inactive ligands datasetThe analysis of 1UTS HIV-1 TAR interactions with its active/inactive ligands was performed on dataset built upon extensive literature search. The details of it's preparation are available in publication *fingeRNAt - a software tool for analysis of nucleic acid-ligand complexes.*  2. Perform molecular docking of active/inactive ligands to 1UTSEach ligand was docked to 1UTS using three different molecular docking programs:* rDock (version 2013.1)* iDock (version 2.2.1)* Dock6 (version 6.9)Results obtained from iDock and Dock6 were converted to sdf using OpenBabel.For each ligand it's three best docked poses (best scores) were used for further analyses. We assume the following architecture of the directory with molecular docking results:```└── docking    ├── rdock    │   └── best_3.sdf    ├── idock    │   └── sdfs    ├── dock6    │   └── best_3.mol2    └── 1UTS.pdb```where* `rdock` and `dock6` directories contain 3 best poses of each ligand in one sdf file * `idock` directory contains subdirectory `sdfs` with 3 best poses of each ligand saved into separate sdf files (each sdf holds 3 poses of one ligand; they were converted from separate pdbqt files)* `1UTS.pdb` is a file with experimental structure of HIV-1 TAR---**It is critical to assure that ligands' poses from molecular docking are ordered from best to worst**, meaning 1st pose of the ligand in file has best score etc.---The details of performed molecular docking can be found in publication *fingeRNAt - a software tool for analysis of nucleic acid-ligand complexes.*  3. Calculate Structural Interactions Fingerprints (SIFs)  3.1. Provide path to molecular docking resultsSimply write it down in the field below e.g. `/Users/ns/Desktop/data`  **Do not hit SHIFT + ENTER after writing it down in the field below.**<jupyter_code>from ipywidgets import Text, HBox, Label
from IPython.display import display

db = HBox([Label('Full path to molecular docking results'), Text()])
display(db)
docking_results = db.children[1].value # get path to molecular docking results from Text object<jupyter_output><empty_output><jupyter_text>3.2. Prepare directories to save SIFs<jupyter_code>%%bash
mkdir -p SIFs_outputs
mkdir -p SIFs_outputs/3best
mkdir -p SIFs_outputs/3best/rdock
mkdir -p SIFs_outputs/3best/dock6<jupyter_output><empty_output><jupyter_text>3.3. Calculate Structural Interactions Fingerprints (SIFs) using program fingeRNAtRun fingeRNAt on 1UTS and each pose of docked ligands from  HIV-1 TAR active/inactive ligands dataset  to calculate SIFt type `FULL`.---**Do not forget to call fingeRNAt inside it's environment!**To do this, you have to run this jupyter notebook from within it. If you did not do that1. Close this notebook2. Open terminal 3. In terminal change directory to `fingeRNAt-additional/Active_vs_inactive_ligands` according to your saved location of the repository4. In terminal type `conda activate fingernat`5. In terminal type `jupyter notebook Pipeline.ipynb` ---See manual for more details about fingeRNAt [installation](https://github.com/n-szulc/fingeRNAtinstallation) and [usage](https://github.com/n-szulc/fingeRNAtusage)  3.3.1. Provide path to program fingeRNAtSimply write it down in the field below e.g. `PATH/TO/DIR/fingeRNAt/code/fingeRNAt.py`.  **Do not hit SHIFT + ENTER after writing it down in the field below.**<jupyter_code>code = HBox([Label('Full path to fingeRNAt.py'), Text()])
display(code)
fingeRNAt_path = code.children[1].value # get path to fingeRNAt.py from Text object<jupyter_output><empty_output><jupyter_text>3.3.2. Run fingeRNAt on 1UTS - rdock results<jupyter_code>%%bash -s "$docking_results" "$fingeRNAt_path"

python $2 -r $1/1UTS.pdb -l $1/rdock/best_3.sdf -o SIFs_outputs/3best/; 
mv SIFs_outputs/3best/1UTS.pdb_best_3.sdf_FULL.tsv SIFs_outputs/3best/rdock.tsv<jupyter_output>#########################                            
                           # Welcome to fingeRNAt! #                            
                           #########################                            

FULL results saved successfully!<jupyter_text>3.3.3. Run fingeRNAt on 1UTS - dock6 results<jupyter_code>%%bash -s "$docking_results" "$fingeRNAt_path"

python $2 -r $1/1UTS.pdb -l $1/dock6/best_3.sdf -o SIFs_outputs/3best/;
mv SIFs_outputs/3best/1UTS.pdb_best_3.sdf_FULL.tsv SIFs_outputs/3best/dock6.tsv<jupyter_output>#########################                            
                           # Welcome to fingeRNAt! #                            
                           #########################                            

FULL results saved successfully!<jupyter_text>3.3.4. Run fingeRNAt on 1UTS - idock results  3.3.4.1 Prepare directory to temporarily store calculated SIFs for idock resultsSince idock outputs for each docked ligand are in separate sdf files, it is neccessary to run fingeRNAt on each 1UTS-ligand separately, unlike as for rdock & dock6 results, which were all saved to one file.<jupyter_code>%%bash
mkdir -p SIFs_outputs/tmp_idock<jupyter_output><empty_output><jupyter_text>3.3.4.2 Run fingeRNAt on each 1UTS-ligand complex<jupyter_code>%%bash -s "$docking_results" "$fingeRNAt_path"

for i in `ls $1/idock/sdfs/`; do python $2 -r $1/1UTS.pdb -l $1/idock/sdfs/$i -o SIFs_outputs/tmp_idock/; done<jupyter_output>#########################                            
                           # Welcome to fingeRNAt! #                            
                           #########################                            

FULL results saved successfully!

                           #########################                            
                           # Welcome to fingeRNAt! #                            
                           #########################                            

FULL results saved successfully!

                           #########################                            
                           # Welcome to fingeRNAt! #                            
                           #########################                            

FULL results saved successfully!

                           #########################                            
                           # Welcome to fingeRNAt! #                            
   [...]<jupyter_text>3.3.4.3 Merge calculated SIFs into one file<jupyter_code>%%bash

# prepare file with headings
cat SIFs_outputs/tmp_idock/`ls SIFs_outputs/tmp_idock/ | tail -n +2 | head -1` | head -n 1 > SIFs_outputs/3best/idock.tsv;

# merge calcuated SIFs into one file
for i in `ls SIFs_outputs/tmp_idock/`; do cat SIFs_outputs/tmp_idock/$i | tail -n +2 SIFs_outputs/tmp_idock/$i >> SIFs_outputs/3best/idock.tsv; done;<jupyter_output><empty_output><jupyter_text>3.3.4.4 Delete `SIFs_outputs/tmp_idock`<jupyter_code>%%bash
rm -r SIFs_outputs/tmp_idock<jupyter_output><empty_output><jupyter_text>4. Preprocess calculated SIFs  4.1. Prepare directories to save preprocessed SIFs<jupyter_code>%%bash
mkdir -p SIFs_outputs/1best
mkdir -p SIFs_outputs/with_activities
mkdir -p SIFs_outputs/with_activities/0
mkdir -p SIFs_outputs/with_activities/1<jupyter_output><empty_output><jupyter_text>4.2. Create files with SIFs results for the declared number of best ligands' poses  4.2.1. Define proper functionDefine function `filter_best_hits` that:* filters declared number of best hits (poses) for each ligand from SIFs results file* saves the results to new tsv file<jupyter_code>import pandas as pd

def filter_best_hits(input_file, hit, output_file):
    '''Filter declared number of best hits (poses) for each ligand from SIFs results file, 
    and save the results to new tsv file'''
    
    df = pd.read_csv(input_file, sep='\t')

    # Create columns with ligand name and ligand pose number separately
    df['Lig_pose'] = df['Ligand_name'].map(lambda x: int(x.split('^')[1]))
    df['Ligand_name'] = df['Ligand_name'].map(lambda x: str(x.split('^')[0]))

    # Group by ligand name and sort by pose number (best score = 1st pose)
    df2 = df.groupby('Ligand_name').apply(lambda x: x.sort_values(["Lig_pose"], ascending = True)).reset_index(drop=True)
    df2 = df2.drop('Lig_pose', axis=1)

    # Get n best hits
    df3 = df2.groupby('Ligand_name').head(hit).reset_index(drop=True)

    df3.to_csv(output_file, sep='\t', index=False)<jupyter_output><empty_output><jupyter_text>4.2.2. Run `filter_best_hits`We already have calculated SIFs for three best poses of each ligand, but we would also like to have calculated SIFs for one best pose of each ligand.However, given inputs with calculated SIFs for more than three best poses, it would be possible to use `filter_best_hits` function to filter any number of best declared ligand's poses from SIFs results file and save them.  4.2.2.1. Run `filter_best_hits` to filter one best pose of each ligand and save their SIFs<jupyter_code>filter_best_hits("SIFs_outputs/3best/rdock.tsv", 1, "SIFs_outputs/1best/rdock.tsv")
filter_best_hits("SIFs_outputs/3best/dock6.tsv", 1, "SIFs_outputs/1best/dock6.tsv")
filter_best_hits("SIFs_outputs/3best/idock.tsv", 1, "SIFs_outputs/1best/idock.tsv")<jupyter_output><empty_output><jupyter_text>4.3. Add information about each ligand's activity and save the results  4.3.1. Define proper funtionDefine function `add_activity` that:* adds information whether each ligand is considered as active/inactive based on prepared `ligands_activity.csv`* save the results to new tsv files<jupyter_code>import pandas as pd

def add_activity(input_file, activity_file, activity, output_file, idock=False):
    '''Add information about each ligand's activity to SIFs input file and save the results to new tsv file'''

    df_a = pd.read_csv(input_file, sep='\t')
    df_a['Ligand_name'] = df_a['Ligand_name'].map(lambda x: str(x.split('^')[0]))
    
    if idock: # delete .pdbqt from ligand name in idock SIFs results
        df_a["Ligand_name"] = df_a["Ligand_name"].str.split('.').str[0]

    activity_df = pd.read_csv(activity_file)
    activity_df = activity_df.drop(['molecule structure', 'type'], axis=1)
    activity_df.rename(columns={'name':'Ligand_name'}, inplace=True)

    df_a2 = pd.merge(df_a, activity_df, on='Ligand_name')

    df_a2 = df_a2[df_a2['activity'] == activity]

    # If you want to save without activity info
    #df_a2 = df_a2.drop(['activity'], axis=1)

    df_a2.to_csv(output_file, sep='\t', index=False)<jupyter_output><empty_output><jupyter_text>4.3.2. Run `add_activity`  4.3.2.1. Run `add_activity` on files with SIFs results of one best pose of each ligand<jupyter_code>for activity in [0,1]:
    add_activity("SIFs_outputs/1best/rdock.tsv", 'input_data/ligands_activity.csv', activity, 'SIFs_outputs/with_activities/{}/rdock_1best.tsv'.format(str(activity)))
    add_activity("SIFs_outputs/1best/dock6.tsv", 'input_data/ligands_activity.csv', activity, 'SIFs_outputs/with_activities/{}/dock6_1best.tsv'.format(str(activity)))
    add_activity("SIFs_outputs/1best/idock.tsv", 'input_data/ligands_activity.csv', activity, 'SIFs_outputs/with_activities/{}/idock_1best.tsv'.format(str(activity)), idock=True)<jupyter_output><empty_output><jupyter_text>4.3.2.2. Run `add_activity` on files with SIFs results of three best poses of each ligand<jupyter_code>for activity in [0,1]:
    add_activity("SIFs_outputs/3best/rdock.tsv", 'input_data/ligands_activity.csv', activity, 'SIFs_outputs/with_activities/{}/rdock_3best.tsv'.format(str(activity)))
    add_activity("SIFs_outputs/3best/dock6.tsv", 'input_data/ligands_activity.csv', activity, 'SIFs_outputs/with_activities/{}/dock6_3best.tsv'.format(str(activity)))
    add_activity("SIFs_outputs/3best/idock.tsv", 'input_data/ligands_activity.csv', activity, 'SIFs_outputs/with_activities/{}/idock_3best.tsv'.format(str(activity)), idock=True)<jupyter_output><empty_output><jupyter_text>5. Summarize results**Summarize SIFs results for active/inactive ligands to see whether there are any differences in binding modes with their molecular target.**---SIFs final results files are in `fingeRNAt-additional/1UTS_HIV-1_TAR_active_inactive_ligands_interactions/SIFs_outputs/with_activities` They are divided by:* ligands' activities - SIFs files contain either active or inactive ligands - subdirectories `1/` and `0/` respectively* docking program that was used to dock ligands - first part of SIFs output file name, which is rdock, dock6 or idock* number of ligand's best poses in SIFs file - second part of SIFs output file name, which is 1best or 3beste.g. `SIFs_outputs/with_activities/0/rdock_1best.tsv`means this file contains SIFs for one best pose of each inactive ligand docked using rDockor`SIFs_outputs/with_activities/1/idock_3best.tsv`means this file contains SIFs for three best poses of each active ligand docked using iDock---The final step of this pipeline is to calculate arithmetic mean for each column from each SIFs results files.  5.1. Calculate arithmetic means for each interaction type with each RNA residue   5.1.1. Prepare directories to save final results<jupyter_code>%%bash
mkdir -p results/rdock
mkdir -p results/dock6
mkdir -p results/idock<jupyter_output><empty_output><jupyter_text>5.1.2. Define proper functionDefine function `calc_residue_interaction_means` that:* calculates arithmetic means for each column of SIFs result file* saves results to new tsv, but with only two columns: residue/interaction id and it's calculated arithmetic mean<jupyter_code>import pandas as pd

interaction_names = {'HB':'Hydrogen Bonds', 'HAL':'Halogen Bonds', 'CA':'Cation-Anion', 'Pi_Cation':'Pi-Cation', 
                     'Pi_Anion':'Pi-Anion', 'Pi_Stacking':'Pi-Stacking', 'Mg_mediated':'Mg-mediated', 
                     'K_mediated':'K-mediated', 'Na_mediated':'Na-mediated', 'Other_mediated':'Other ion-mediated',
                     'Water_mediated':'Water-mediated', 'Lipophilic':'Lipophilic'}

def calc_residue_interaction_means(activities, poses, software):
    '''Calculate arithmetic mean of SIFs for each column from SIFs result file'''

    for activity in activities:
        
        for pose in poses:
            slownik = {}

            df_interact = pd.read_csv('SIFs_outputs/with_activities/{}/{}_{}best.tsv'.format(activity, software, pose), sep='\t')
            df_interact = df_interact.drop(['Ligand_name', 'activity'], axis=1)
            cols = list(df_interact.columns)
            df_interact2 = df_interact.mean(axis = 0)

            for c in cols:
                res_id = c.split('#')[1]
                no = res_id.split(':')[0]
                chain = res_id.split(':')[1]
                interaction = interaction_names[c.split('#')[2]]
                c1 = '{}:{}:{}'.format(interaction, no, chain)

                if c1 not in slownik.keys():
                    slownik[c1] = []
                slownik[c1].append(round(df_interact2[c],3))

            ALL_DF = pd.DataFrame.from_dict(slownik, orient='index', columns=[software])
            ALL_DF.reindex(index=ALL_DF.index[::-1])
            ALL_DF_t = ALL_DF.transpose()

            ALL_DF.to_csv('results/{}/{}_{}best.tsv'.format(software, activity, pose), sep='\t')<jupyter_output><empty_output><jupyter_text>5.1.3. Run `calc_residue_interaction_means`<jupyter_code>activities = ['0','1']
poses = ['1', '3']

for soft in ['rdock', 'dock6', 'idock']:
    calc_residue_interaction_means(activities, poses, soft)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11652019978504141, 'fraction_numerical': 0.01605867105013593, 'mean_word_length': 3.661951075744179, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 73, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 7, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '73351', 'license': 'Apache-2.0', 'n_tokens_mistral': 5321, 'n_tokens_neox': 5007, 'n_words': 1420, 'path': 'HIV-1_TAR_active_vs_inactive_ligands/Pipeline.ipynb', 'repo_name': 'n-szulc/fingeRNAt-supplementary', 'size': '15817'}