<jupyter_start><jupyter_text>*Drosophila melanogaster* Genetic Interactions<jupyter_code>import matplotlib
matplotlib.use('Agg')
%matplotlib inline
import matplotlib.pyplot as plt
import bisect
import itertools
import numpy
import os
import pickle<jupyter_output><empty_output><jupyter_text>---  Extracting Genetic Interactions from BIOGRIDDownloaded and starting using *BIOGRID-ORGANISM-3.3.124.tab2.zip* on 13-14 May 2015. Using *BIOGRID-ORGANISM-3.4.127.tab2* on 4 August 2015.Download version 3.4.130 on 3 November 2015, started using on 30 December 2015 for *D. melanogaster*.Header line looks like:<jupyter_code>%%bash
head -1 /work/jyoung/DataDownload/BIOGRID/BIOGRID-ORGANISM-3.3.124.tab2/BIOGRID-ORGANISM-Drosophila_melanogaster-3.3.124.tab2.txt<jupyter_output>#BioGRID Interaction ID	Entrez Gene Interactor A	Entrez Gene Interactor B	BioGRID ID Interactor A	BioGRID ID Interactor B	Systematic Name Interactor A	Systematic Name Interactor B	Official Symbol Interactor A	Official Symbol Interactor B	Synonyms Interactor A	Synonyms Interactor B	Experimental System	Experimental System Type	Author	Pubmed ID	Organism Interactor A	Organism Interactor B	Throughput	Score	Modification	Phenotypes	Qualifications	Tags	Source Database<jupyter_text>Data lines look like:<jupyter_code>%%bash
head -2 /work/jyoung/DataDownload/BIOGRID/BIOGRID-ORGANISM-3.3.124.tab2/BIOGRID-ORGANISM-Drosophila_melanogaster-3.3.124.tab2.txt | tail -1<jupyter_output>30772	43383	37006	68259	62692	Dmel_CG10002	Dmel_CG6459	fkh	P32	CG10002|DmelCG10002|Dmfkh|Sebp2|dFoxA	CG 6459|CG6459|DmelCG6459|dp32	Two-hybrid	physical	Giot L (2003)	14605208	7227	7227	High Throughput	-	-	-	-	-	BIOGRID<jupyter_text>Filter lines that have "Experimental System Type" as "genetic":<jupyter_code>readPath = os.path.join('..', '..', 'DataDownload', 'BIOGRID', 'BIOGRID-ORGANISM-3.4.127.tab2', 
                        'BIOGRID-ORGANISM-Drosophila_melanogaster-3.4.127.tab2.txt')
writePath = os.path.join('..', 'data','BIOGRID-3.4.127-fly.txt')
header = open(readPath).readline()
colNum = header.strip().split('\t').index('Experimental System Type')
with open(writePath, 'w') as fid:
    fid.write(header)
    fid.writelines([line for line in open(readPath) if line.split('\t')[colNum] == 'genetic'])
%%bash
wc -l /work/jyoung/genetic_interact/data/BIOGRID-3.4.127-fly.txt<jupyter_output>9979 /work/jyoung/genetic_interact/data/BIOGRID-3.4.127-fly.txt<jupyter_text>Breakdown of experimental evidence codesUsing *BIOGRID-ORGANISM-3.3.124.tab2* on 10 June 2015.Using *BIOGRID-ORGANISM-3.4.127.tab2* on 4 August 2015.<jupyter_code>os.chdir('/work/jyoung/genetic_interact/data')
catg = dict()
genes = set()
for line in open('BIOGRID-3.4.127-fly.txt'):
    tokens = line.split('\t')
    genes.update(tokens[1:3])
    catg[tokens[11]] = catg.get(tokens[11], 0) + 1
len(genes)
catg<jupyter_output><empty_output><jupyter_text>FlyNet**2015 May 13-15**Downloaded and started using [FlyNet](http://www.inetbio.org/flynet/downloadnetwork.php).<jupyter_code>%%bash
head -5 /work/jyoung/DataDownload/FunctionalNet/FlyNet.txt<jupyter_output>FBgn0000588	FBgn0000629	5.9845282658116
FBgn0004406	FBgn0004407	5.98385558064729
FBgn0039907	FBgn0043900	5.97562686691344
FBgn0000352	FBgn0001079	5.96739494043322
FBgn0000499	FBgn0002945	5.96444550034175<jupyter_text>So FlyNet uses FlyBase IDs. These will be converted to Entrez IDs for use with the BIOGRID genetic interactions. Note that the FlyBase conversion tool doens't actually convert to Entrez IDs. DAVID seems to only recognize 12138 of the FlyBase IDs. BioMart only "advises" conversion of 500 IDs at a time. Using the R Bioconductor library *org.Dm.eg.db*, an R script *convert_flynet_entrez.R* was written to convert the FlyBase IDs in FlyNet to Entrez IDs. Not all IDs were able to be converted; the FlyBase IDs without a conversion are labeled NULL. The output file is *FlyNetEntrez.txt*.  **2015 June 11**Create a new file, *FlyNetEntrez-noNull.txt*, that has any line with NULL entries removed.<jupyter_code>%%bash
sed '/NULL/d' FlyNetEntrez.txt > FlyNetEntrez-noNull.txt<jupyter_output><empty_output><jupyter_text>Predictability of Each BIOGRID Interaction Type**2015 June 11**Using *BIOGRID-ORGANISM-3.3.124.tab2*.**2015 August 4**Using *BIOGRID-ORGANISM-3.4.127.tab2*. **2015 August 4-7**Major updates to *func_net_pred.py*. It no longer requires working only with Entrez IDs. The *get_entrez_indices( )* function has been replaced with the *assign_gene_indices( )* function. Sorting of gene IDs still occurs so that every time the script is run, the indices of the adjacency matrix always correspond to the same gene IDs. The *write_seeds_text( )* function no longer does a gene ID conversion. The *main( )* function contains a flag variable that indicates whether or not any conversion needs to be done. Note that the conversion should be based from the functional network; that is, the conversion dictionary should be built from converting the genes in the network. The updated version of *func_net_pred.py* was tested to ensure that it produces the same plots and seed set text output as before.In an ideal scenario, the functional network and the BIOGRID file use the same gene IDs. If not, then convert to Entrez, as it seems to have less ambiguity than either official symbols or systematic name. For example, the R conversion has names such as "CG6646 gene product from transcript CG6646-RA", while BIOGRID simply uses "CG6646".<jupyter_code>os.chdir('/work/jyoung/genetic_interact/src')
%run func_net_pred 'Dosage Rescue'
%run func_net_pred 'Synthetic Haploinsufficiency'
%run func_net_pred 'Phenotypic Enhancement'
%run func_net_pred 'Phenotypic Suppression'
%run func_net_pred 'Negative Genetic'
%run func_net_pred 'Synthetic Rescue'
%run func_net_pred 'Synthetic Growth Defect'
%run func_net_pred 'Synthetic Lethality'<jupyter_output>Number of genes in functional network: 13052
Number of genes in interactions: 4
Number of seed sets: 4<jupyter_text>Output Gene Names in Predictable Seed Sets**2015 July 2-3**To examine the predictable seed sets (those having high AUC), write out the gene names of the seeds and their interactors. The fly Entrez IDs will need to be converted to fly gene names. Use the R Bioconductor library *org.Dm.eg.db* to output a conversion table to text, which can be read and stored in Python.     > library("org.Dm.eg.db")    > entrezIDs <- ls(org.Dm.egGENENAME)    > length(entrezIDs)    [1] 24620    > entrezIDs[1:5]    [1] "30970" "30971" "30972" "30973" "30975"    > head(toTable(org.Dm.egGENENAME[entrezIDs]))      gene_id                                       gene_name    1   30970   CG3038 gene product from transcript CG3038-RC    2   30971   CG2995 gene product from transcript CG2995-RB    3   30972 CG13377 gene product from transcript CG13377-RB    4   30973                                        cinnamon    5   30975                                      erect wing    6   30976 CG13375 gene product from transcript CG13375-RB    > conversTab <- toTable(org.Dm.egGENENAME[entrezIDs])    > write.table(conversTab, file="/work/jyoung/DataProcessed/org.Dm.egGENENAME.txt",                  quote=FALSE, sep="\t", row.names=FALSE)Next, verify that the table has unique gene IDs and read this conversion table into a Python dictionary.<jupyter_code>%%bash
head -4 /work/jyoung/DataProcessed/org.Dm.egGENENAME.txt
%%bash
cut -f1 /work/jyoung/DataProcessed/org.Dm.egGENENAME.txt | sort | uniq | wc -l
entrez2name = dict()
fid = open('/work/jyoung/DataProcessed/org.Dm.egGENENAME.txt')
header = fid.readline()
for line in fid:
    tokens = line.strip().split('\t')
    entrez2name[tokens[0]] = tokens[1]
fid.close()<jupyter_output><empty_output><jupyter_text>Now check whether all of the Entrez IDs in FlyNet are included in the conversion dictionary.<jupyter_code>allNetEntrez = set()
for line in open('../data/FlyNetEntrez-noNull.txt'):
    allNetEntrez.update(line.split('\t')[:2])
len(allNetEntrez)
len(allNetEntrez - set(entrez2name.keys()))
pickle.dump(entrez2name, open('/work/jyoung/PyPickle/flyentrez2names.p', 'wb'))<jupyter_output><empty_output><jupyter_text>Modifications to the *write_seeds_text( )* function (which was originally from *pred_from_huNet_v2.py*):- changed the number of seed sets written to be based on AUC score instead of an arbitrary numerical cutoff- changed the filepath for Entrez-to-name conversion from */work/jyoung/PyPickle/entrez2names.p* to *flyentrez2names.p*- *entrez2names* converts an Entrez ID to its multiple gene names, the 1st of which is commonly used. On the other hand, *flyentrez2names* converts a single Entrez ID to a single gene name. Therefore, any *entrez2names[ ][0]* code is changed to *entrez2names[ ]*. Writing out seed sets for Phenotypic Enhancement (using v3.4.127 on 2015-08-04):<jupyter_code>%run func_net_pred 'Phenotypic Enhancement'
%run func_net_pred 'Phenotypic Suppression'
%run func_net_pred 'Synthetic Rescue'
%run func_net_pred 'Synthetic Growth Defect'<jupyter_output>Number of genes in functional network: 13052
Number of genes in interactions: 5
Number of seed sets: 5<jupyter_text>Cytoscape Network Visualization**2015 July 6**To visualize the predictive seed sets in Cytoscape, write out the interacting genes in SIF format. The plan is to use the ClueGO app, which accepts gene IDs. So the text file with the seed sets will not only need to be re-written in SIF format, but with Entrez IDs.<jupyter_code>os.chdir('/work/jyoung/genetic_interact/src')
import func_net_pred
experimentSys = 'Phenotypic Enhancement'
# copied from main()
node2edgewt = func_net_pred.process_func_net()
entrez2idx = func_net_pred.get_entrez_indices(node2edgewt)
adjMat = numpy.load('../data/FlyNet_adj_matrix.npy')
seedSets = func_net_pred.read_biogrid(experimentSys)
seedAUC, seed2interactors = func_net_pred.seed_set_predictability(entrez2idx, adjMat, seedSets)
def write_sif(seed2interactors, seedAUC, writeFilename):
    """
    Adapted from write_seed_sets(). Writes seed sets into SIF format for input 
    into Cytoscape. 'gi' is genetic interaction for the <relationship type> 
    ARGUMENTS:
        1.) <dict> {seed Entrez in func net: {interactors' Entrez in func net}}
        2.) <list> [(seed AUC seed Entrez)]
        3.) <string> output filename
    """
    AUCcutoff = 0.9
    aucs = [v[0] for v in seedAUC]
    cutIndex = len(aucs) - bisect.bisect_left(aucs, AUCcutoff)
    writeFile = open(writeFilename, 'w')
    for i in range(1, cutIndex+1):
        intactEntrez = list(seed2interactors[seedAUC[-i][1]])
        if len(intactEntrez) > 1:
            writeFile.write(seedAUC[-i][1] + '\tgi\t' + '\t'.join(intactEntrez) + '\n')
    writeFile.close()
os.chdir('/work/jyoung/genetic_interact/results')
write_sif(seed2interactors, seedAUC, 'PhenotypicEnhancement_seed_entrez.sif')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09627988553493953, 'fraction_numerical': 0.05594018277485461, 'mean_word_length': 4.818474758324382, 'pattern_counts': {'":': 1, '<': 49, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '104874', 'license': 'MIT', 'n_tokens_mistral': 4149, 'n_tokens_neox': 3592, 'n_words': 1079, 'path': 'doc/Fly Genetic Interactions.ipynb', 'repo_name': 'jon-young/genetic_interact', 'size': '10833'}