<jupyter_start><jupyter_text>Database construction for FIA-MS This example notebook shows how to create the organism specific database files (structure and mapping) necessary for processing FIA-MS data. The files are based on metabolite lists and additional information that can be taken from metabolic model. Here we present the processing steps necessary for processing models saved as three different file types; .xlsx, .json and .sbml.<jupyter_code>import pandas as pd
import cobra
from collections import namedtuple
import BFAIR.FIA_MS as fia_ms<jupyter_output>Determination of memory status is not supported on this 
 platform, measuring for memoryleaks will never fail<jupyter_text>Organism specific databases  Yeast The yeast model was described [here](https://www.nature.com/articles/s41467-019-11581-3) and can be found on [this website](https://sysbiochalmers.github.io/yeast-GEM/).<jupyter_code>yeast_df = pd.read_excel("data/FIA_MS_example/database_files/yeastGEM.xlsx", sheet_name='METS', engine='openpyxl')
# First, we reduce the list of metabolites. They are listed for each compartment in the model but we only need them once
yeast_df_unique = yeast_df.drop_duplicates(subset='NAME', keep='first')
# Then we kick out metabolites that do not have an annotated composition/formula
yeast_df_unique = yeast_df_unique.dropna(subset=['COMPOSITION'])
# And finally we remove metabolites that are connected to a "rest" of a molecule or a halogene
yeast_df_unique = yeast_df_unique[~yeast_df_unique['COMPOSITION'].str.contains('R')]
yeast_df_unique = yeast_df_unique[~yeast_df_unique['COMPOSITION'].str.contains('X')]
# Check how many metabolites and how many unique structures remain
len(yeast_df_unique), len(yeast_df_unique.COMPOSITION.unique())
# Reformat the dataframe so that it contains the necessary information
Metabolite = namedtuple('Metabolite', ['id', 'formula', 'charge', 'name'])
yeast_mets = [
    Metabolite(id=row['REPLACEMENT ID'], formula=row['COMPOSITION'], charge=row['CHARGE'], name=row['NAME'])
    for i, row in yeast_df_unique.iterrows()]
fia_ms.create_database(yeast_mets, 'yeastGEM', 'data/FIA_MS_example/database_files/CHEMISTRY')<jupyter_output><empty_output><jupyter_text>E. coli The E. coli model was described [here](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3261703/) and can be found on [this website](http://bigg.ucsd.edu/models/iJO1366).<jupyter_code>model_coli = cobra.io.load_json_model('data/FIA_MS_example/database_files/iJO1366.json')
fia_ms.create_database(model_coli.metabolites, 'iJO1366', 'data/FIA_MS_example/database_files/CHEMISTRY')<jupyter_output><empty_output><jupyter_text>P. putida The P. putida model was described [here](https://sfamjournals.onlinelibrary.wiley.com/doi/full/10.1111/1462-2920.14843) and can be found on [this website](http://bigg.ucsd.edu/models/iJN1463).<jupyter_code>model_putida = cobra.io.load_json_model('data/FIA_MS_example/database_files/iJN1463.json')
fia_ms.create_database(model_putida.metabolites, 'iJN1463', 'data/FIA_MS_example/database_files/CHEMISTRY')<jupyter_output><empty_output><jupyter_text>Streptomyces The Streptomyces metabolite lists were set up in house.<jupyter_code># import the streptomyces metabolic model from an excel file
df_mets = pd.read_excel('data/FIA_MS_example/database_files/449_2018_1900_MOESM1_ESM.xlsx', sheet_name='Metabolites', skiprows=1, engine='openpyxl')
# Here, such as for the yeast database, we need to pick out the relevant information from an excel file
# first we initialize lists to write our metabolite information intra- and extra cellular metabolites in to
mets_endo = []
mets_exo = []
# then we initialize the type of object we're going to use for the metabolite information
Metabolite = namedtuple('Metabolite', ['id', 'formula', 'charge', 'name'])
# we iterate through the imported dataframe and pick out the relevant information
# we exclude components that are listed as 'biomass' components in their description and metabolites
# that have non-integer coefficients in their metabolite formulas
for i, row in df_mets.iterrows():
    desc = row['Metabolite description']
    if 'biomass' in desc.lower() or 'Acyl_sn_glycerol_3_phosphate_C18.925H37.908O7P' == desc:
        continue
    if '.' in row['Metabolite formula']:
        continue
    m = Metabolite(
        id=row['Metabolite name'], 
        formula=row['Metabolite formula'], 
        charge=0, 
        name=row['Metabolite description'],
    )
    # separate intra- and extra cellular metabolites depending on their tags
    if m.id[-3:] == '[e]':
        mets_exo.append(m)
    else:
        mets_endo.append(m)
# check the lengths of the generated lists
len(mets_endo), len(mets_exo)
# import information about possible secondary metabolites from a separate excel file
df_secondary = pd.read_excel('data/FIA_MS_example/database_files/secondary_collinus.xlsx', engine='openpyxl')
# again, iterate through the imported dataframe and pick out the relevant information
# we exclude components that have non-integer coefficients in their metabolite formulas
for i, row in df_secondary.iterrows():
    if isinstance(row['Formula'], float):
        continue
    m = Metabolite(
        id=row['Name'].replace(' ', ''), 
        formula=row['Formula'], 
        charge=0, 
        name=row['Name'],
    )
    mets_exo.append(m)
    mets_endo.append(m)
# check the lengths of the appended lists
len(mets_endo), len(mets_exo)
fia_ms.create_database(mets_endo, 'streptomyces_endo', 'data/FIA_MS_example/database_files/CHEMISTRY')
fia_ms.create_database(mets_exo, 'streptomyces_exo', 'data/FIA_MS_example/database_files/CHEMISTRY')<jupyter_output><empty_output><jupyter_text>C. elegans The C. elegans model was described [here](https://doi.org/10.3389/fmolb.2019.00002) and can be found on [this website](https://figshare.com/articles/dataset/Data_Sheet_2_Multi-Omics_and_Genome-Scale_Modeling_Reveal_a_Metabolic_Shift_During_C_elegans_Aging_ZIP/7679876/1).<jupyter_code>model_celegans = cobra.io.read_sbml_model('data/FIA_MS_example/database_files/wormjam-20180125.sbml')
fia_ms.create_database(model_celegans.metabolites, 'Celegans', 'data/FIA_MS_example/database_files/CHEMISTRY')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10299324106855487, 'fraction_numerical': 0.019954940457032506, 'mean_word_length': 5.897891231964484, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '280712', 'license': 'MIT', 'n_tokens_mistral': 2168, 'n_tokens_neox': 1986, 'n_words': 506, 'path': 'docs/examples/FIA_MS_database_construction_example.ipynb', 'repo_name': 'biosustain/AnalyticsTools', 'size': '6214'}