<jupyter_start><jupyter_text>Analyse predictions by cTaG2This notebook is used to analyse predictions made by cTaG2 for LUAD data.<jupyter_code>import os
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output>In /home/malvika/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The text.latex.preview rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/malvika/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The mathtext.fallback_to_cm rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/malvika/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: Support for setting the 'mathtext.fallback_to_cm' rcParam is deprecated since 3.3 and will be removed two minor releases later; use 'mathtext.fallback : 'cm' instead.
In /home/malvika/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The validate_bool_maybe_none function was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/malvika/anaconda3/lib/python3.7/site-packages/m[...]<jupyter_text>Load the prediction fileVariables to be updated for loading the prediction file.Either the absolute path is given to *filepath* or set *PATH* and the relative path is set based on cancer type, label method, number of features and model. Default file name is "cTaG2_predictions.tsv". For other filenames, update *fname*.<jupyter_code>PATH = "/data/malvika/cTaG2.0"
ctype = "LUAD"
lab_type = "bailey"
feat_num = "some"
model = "BalBag"
folderpath = "/output/GDC_{}/predict/multiomic".format(ctype)

filepath = PATH + folderpath + "/{}_{}_{}".format(lab_type, feat_num, model)
## Uncomment line below to set absolute path
# filepath = "\path\to\prediction\file"
os.chdir(filepath)
fname = "cTaG2_predictions.tsv"
data = pd.read_csv(fname, sep="\t", header=0, index_col=0)
data.head(5)
print("Number of total samples = {}".format(len(data["Tumor_Sample_Barcode"].unique())))
print("Total number of unique genes = {}".format(len(data["Hugo_Symbol"].unique())))
print("Total number of unique TSG = {}".format(list(data[["Hugo_Symbol", "Predicted label"]].drop_duplicates()["Predicted label"]).count("Tumor suppressor")))
print("Total number of unique OG = {}".format(list(data[["Hugo_Symbol", "Predicted label"]].drop_duplicates()["Predicted label"]).count("Oncogene")))
data_unigenes = data[["Hugo_Symbol", "Predicted label"]].drop_duplicates()
multi_genes = [gene for gene in data_unigenes["Hugo_Symbol"].unique() if data_unigenes[data_unigenes["Hugo_Symbol"]==gene].shape[0]>1] 
len(multi_genes)<jupyter_output><empty_output><jupyter_text>Summary statistics for sampleCalculates the total number of genes predicted for each sample along with other counts.Counts the number of genes with mutation, and CNV alterations.Counts number of TSGs and OGs predicted for the sample.<jupyter_code>data_samp = pd.DataFrame()
data_samp["Tumor_Sample_Barcode"] = data["Tumor_Sample_Barcode"].unique()
data_samp["Num_genes"] = [data[data["Tumor_Sample_Barcode"] == samp].shape[0] for samp in data_samp["Tumor_Sample_Barcode"]]
data_samp["Num_mut"] = [data[(data["Tumor_Sample_Barcode"] == samp) & ~(data["Variant_Classification"].isna())].shape[0] for samp in data_samp["Tumor_Sample_Barcode"]]
data_samp["Num_cnv"] = [data[(data["Tumor_Sample_Barcode"] == samp) & ~(data["CNV"] == 0)].shape[0] for samp in data_samp["Tumor_Sample_Barcode"]]
data_samp["Num_TSG"] = [data[(data["Tumor_Sample_Barcode"] == samp) & (data["Predicted label"] == "Tumor suppressor")].shape[0] for samp in data_samp["Tumor_Sample_Barcode"]]
data_samp["Num_OG"] = [data[(data["Tumor_Sample_Barcode"] == samp) & (data["Predicted label"] == "Oncogene")].shape[0] for samp in data_samp["Tumor_Sample_Barcode"]]

data_samp.head()<jupyter_output><empty_output><jupyter_text>Distribution of number of genes predictedThe distribution of number of genes identified for each sample.<jupyter_code>ax = sns.distplot(data_samp["Num_genes"])
# plt.ylim(-0.05, 1)
ax.set_xlabel("Number of genes predicted",fontsize=15)
ax.set_ylabel("Density of samples",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()

# os.chdir(filepath)
# fname = "{}_geneXsamp.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Distribution of number of mutated genes predictedThe distribution of number of genes identified for each sample that were mutated.<jupyter_code>ax = sns.distplot(data_samp["Num_mut"])
ax.set_xlabel("Number of mutated genes",fontsize=15)
ax.set_ylabel("Density of samples",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()

# os.chdir(filepath)
# fname = "{}_mutXsamp.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Distribution of number of genes predicted with CNVsThe distribution of number of genes identified for each sample that showed CNVs.<jupyter_code>ax = sns.distplot(data_samp["Num_cnv"])
ax.set_xlabel("Number of CNV",fontsize=15)
ax.set_ylabel("Density of samples",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()

# os.chdir(filepath)
# fname = "{}_cnvXsamp.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Distribution of number of TSGs predictedThe distribution of number of genes labelled as TSG for each sample.<jupyter_code>ax = sns.distplot(data_samp["Num_TSG"])
ax.set_xlabel("Number of TSGs",fontsize=15)
ax.set_ylabel("Density of samples",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()

# os.chdir(filepath)
# fname = "{}_tsgXsamp.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Distribution of number of OGs predictedThe distribution of number of genes labelled as OG for each sample.<jupyter_code>ax = sns.distplot(data_samp["Num_OG"])
ax.set_xlabel("Number of OGs",fontsize=15)
ax.set_ylabel("Density of samples",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()

# os.chdir(filepath)
# fname = "{}_ogXsamp.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Distribution of degree of genes predictedThe distribution of degree of genes identified for each sample.<jupyter_code>ax = sns.distplot(data["Degree"])
ax.set_xlabel("Degree of a gene",fontsize=15)
ax.set_ylabel("Density of samples",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()

# os.chdir(filepath)
# fname = "{}_degreeXsamp.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Gene based statistics<jupyter_code>data_genes = pd.DataFrame(data.groupby(["Hugo_Symbol"])["Tumor_Sample_Barcode"].count())
data_genes["Degree"] = data.groupby(["Hugo_Symbol"])["Degree"].mean()<jupyter_output><empty_output><jupyter_text>Consensus with CGCLoad CGC data and check for consensus<jupyter_code>os.chdir(PATH + "/data/driver genes/CGC")
fname = "cancer_gene_census_9nov2021.csv"
data_cgc = pd.read_csv(fname, sep=",", header=0, index_col=0)
data_genes.loc[:, "CGC"] = [True if gene in list(data_cgc.index) else False for gene in list(data_genes.index)]
print("Total number of genes showing consensus with CGC = {}".format(data_genes["CGC"].sum()))<jupyter_output>Total number of genes showing consensus with CGC = 188<jupyter_text>Distribution of number of samples a gene is predicted as driverThe distribution of number of samples a gene is identified as driver.<jupyter_code>ax = sns.distplot(data_genes["Tumor_Sample_Barcode"], label="All genes")
sns.distplot(data_genes[data_genes.CGC ==True]["Tumor_Sample_Barcode"], label="CGC genes")
sns.distplot(data_genes[data_genes.CGC ==False]["Tumor_Sample_Barcode"], label="novel genes")
ax.set_xlabel("Number of samples",fontsize=15)
ax.set_ylabel("Density of genes altered",fontsize=15)
ax.tick_params(labelsize=14)
plt.title(ctype, fontsize=16)
plt.tight_layout()
plt.legend()

# os.chdir(filepath)
# fname = "{}_sampXgenes.png".format(ctype)
# plt.savefig(fname, dpi=300)
# plt.close()<jupyter_output><empty_output><jupyter_text>Top 10 genes frequently predictedThe list of genes and the number of samples they are predicted as TSG or OG.<jupyter_code>data_genes.sort_values(by=["Tumor_Sample_Barcode"], ascending=False)[:10]<jupyter_output><empty_output><jupyter_text>Top 10 genes rarely predicted as driverThe list of genes and the number of samples they are predicted as TSG or OG. Random 10 genes are listed below. More genes mutated in one sample may be<jupyter_code>data_genes.sort_values(by=["Tumor_Sample_Barcode"], ascending=True)[:10]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13483770564695421, 'fraction_numerical': 0.015673632725655848, 'mean_word_length': 7.743440233236152, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '309162', 'license': 'CC0-1.0', 'n_tokens_mistral': 3374, 'n_tokens_neox': 3197, 'n_words': 679, 'path': 'code/analyse_predictions_LUAD.ipynb', 'repo_name': 'RamanLab/PIVOT', 'size': '8996'}