<jupyter_start><jupyter_text>First import the txt file exported from DA and convert it to a dictionary for the convenience of later process<jupyter_code>raw_txt_path = r'./SBB5-10cm_mz520-580.txt'

spectra = msi_from_txt(raw_txt_path)<jupyter_output>100%|██████████| 9554/9554 [00:45<00:00, 209.35it/s]<jupyter_text>Secondly, get a list of reference peaks that will be used to align all spectra<jupyter_code>ref = get_ref_peaks(spectra)<jupyter_output>100%|██████████| 59/59 [00:00<00:00, 25179.48it/s]<jupyter_text>Align all spectra onto the reference peaks that were identified in the last step<jupyter_code>feature_table = create_feature_table(spectra, ref)<jupyter_output>Binning the spectrum...<jupyter_text>Here is the head 5 rows of the resulted feature table. The table is organized in such a way, that each row actually represents one spectrum from a single spot, and each column represent one ion image. The last two columns are the x, y coordinates of the corresponding spot in the same row. The cell values are the peak intensity from the txt file, without any normalization.<jupyter_code>feature_table.head(5)<jupyter_output><empty_output><jupyter_text>Then we rank all ion images based on their structure scores<jupyter_code>t_df, ims = get_peak_ranks(feature_table)<jupyter_output>100%|██████████| 776/776 [00:16<00:00, 47.74it/s]<jupyter_text>Take a look at the top 5 ion images as an example what's picked in this step<jupyter_code>import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 5)
ax_idx = 0
for i in list(t_df.values.argsort(0)[-1:-6:-1].flatten()):
    ax[ax_idx].imshow(ims[i])
    ax[ax_idx].set_title(list(t_df.index)[i])
    ax[ax_idx].axis('off')
    ax_idx += 1
plt.show()<jupyter_output><empty_output><jupyter_text>Then the bottom 5 ion images as an example what's dropped in this step<jupyter_code>fig, ax = plt.subplots(1, 5)
ax_idx = 0
for i in list(t_df.values.argsort(0)[0:5:1].flatten()):
    ax[ax_idx].imshow(ims[i])
    ax[ax_idx].set_title(list(t_df.index)[i])
    ax[ax_idx].axis('off')
    ax_idx += 1
plt.show()<jupyter_output><empty_output><jupyter_text>Here we decide 0.4 to be the appropriate cutoff score for peak picking, the value may vary in different experiments<jupyter_code>feature_table, ims = sel_peak_by_rank(t_df, ims, feature_table, 0.4)<jupyter_output><empty_output><jupyter_text>To get a relatively stable NMF result, we need to first determine the rank we want to use.<jupyter_code>rank_candidates = list(range(2, 20, 2))

rank_estimate(rank_candidates, ims)<jupyter_output><empty_output><jupyter_text>Here we choose a rank of 8, because the Cophenetic correlation peaks. The algorithm needs to be run repetitively (n_run > 1) to produce network<jupyter_code>b, c, G = nmf(ims, feature_table, 8, n_run=10)<jupyter_output>C:\Users\hlabuser\PycharmProjects\msi_feature_extraction\mfe\src\feature.py:97: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  consensus.values[[np.arange(len(consensus))] * 2] = np.nan<jupyter_text>Here we can see what the extracted 8 features look like (basis matrix), the contribution of each ion image to each component is stored in the coefficients matrix `c`<jupyter_code>fig, ax = plt.subplots(1, 8)
for i in range(len(b.T)):
    ax[i].imshow(b.iloc[:,i].to_numpy().reshape(ims[0].shape))
    ax[i].set_title(f'{i}')
    ax[i].axis('off')
    ax_idx += 1
plt.show()<jupyter_output><empty_output><jupyter_text>Have a look at what the molecular network looks like<jupyter_code>import networkx as nx
nx.draw(G, node_size=1, pos=nx.spring_layout(G))<jupyter_output><empty_output><jupyter_text>Save the network for other analysis<jupyter_code>nx.write_graphml(G, './SBB5-10cm_520-580_mn.graphml')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11837769328263625, 'fraction_numerical': 0.032192648922686945, 'mean_word_length': 6.361940298507463, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '601089', 'license': 'MIT', 'n_tokens_mistral': 1517, 'n_tokens_neox': 1404, 'n_words': 384, 'path': 'examples/SBB5-10cm_Feature_Extraction.ipynb', 'repo_name': 'weimin-liu/msi_feature_extraction', 'size': '3945'}