<jupyter_start><jupyter_text>Transition map inference Below, we illustrate transition map inference from different methods in cospar. Once computed, they can be analyzed and visualized later.<jupyter_code>import cospar as cs
cs.logging.print_version()
cs.settings.verbosity=2 #range: 0 (error),1 (warning),2 (info),3 (hint). 
cs.settings.set_figure_params(format='png',figsize=[4,3.5],dpi=75,fontsize=14,pointsize=3) 
# Each dataset should have its folder to avoid conflicts.
cs.settings.data_path='data_cospar' 
cs.settings.figure_path='fig_cospar' 
cs.hf.set_up_folders()<jupyter_output><empty_output><jupyter_text>Load an existing dataset. (If you have pre-processed data, you can load it with `cs.hf.read(file_name)`.)<jupyter_code>adata_orig=cs.datasets.hematopoiesis_subsampled()<jupyter_output><empty_output><jupyter_text>Transition map from clones across multiple time points This method utilizes clones with multiple time points. It is the core algorithm for the CoSpar package. The first time it runs, it will compute the similarity matrix at different smooth depth and save them. Then, it will infer the transition map. Some key parameters:    * **clonal_time_points** (`list`; default: all): List of time points with clonal data that you want to perform inference.    * **later_time_point** (`str`; default: None): If not set, CoSpar learns a transition map between neighboring time points specified by clonal_time_points, i.e., transitions from day 2 to day 4, and from day 4 to day 6 if clonal_time_points=['2','4','6']. Ohterwise, it learns transitions from any of `clonal_time_points` to the `later_time_point`.    * **smooth_array** (`list`; default: [15,10,5]): List of smooth depth at initial runs of iteration. Suppose that it has a length N. For iteration nN, we use the last entry of smooth_array to compute the S matrix. We recommend starting with more smoothing depth and gradually reduce the depth, as inspired by simulated annealing. Data with higher clonal dispersion should start with higher smoothing depth. The final depth should depend on the manifold itself. For fewer cells, it results in a small KNN graph, and a small final depth should be used. We recommend to use a number at the multiple of 5 for computational efficiency i.e., smooth_array=[20, 15, 10, 5], or [20,15,10].    * **sparsity_threshold** (`float`; default: 0.1): sparsity threshold to remove spurious transitions in the updated transition map, in the range [0,1].* **max_iter_N** (`int`; default: 5): maximum number of iterations in refining the transition map. We found that 3 iterations are sufficient for tested datasets.* **epsilon_converge** (`float`; default: 0.05): Iteration is considered converged when the correlation R between maps from consecutive iteration satisfies: R>1-`epsilon_converge`.   It takes ~3 mins to run for the first time, and ~20 s for later runs.<jupyter_code>adata=cs.tmap.infer_Tmap_from_multitime_clones(adata_orig,clonal_time_points=['2','4'], 
                    later_time_point='6',smooth_array=[20,15,10,5],
          sparsity_threshold=0.1,intraclone_threshold=0.2,max_iter_N=10,epsilon_converge=0.01)<jupyter_output>Trying to set attribute `.uns` of view, copying.<jupyter_text>We can see that after this step, we have two maps: `'transition_map'` and `'intraclone_transition_map'`. Both of them can be used     for downstream analysis.<jupyter_code>cs.hf.check_available_map(adata)
adata.uns['available_map']<jupyter_output><empty_output><jupyter_text>Transition map from end-point clones, or without clones After initializing the map by either the `OT` or `HighVar` method, We jointly infer the likely clonal ancestors and the corresponding transition map between given time points. You need to choose the initialization method and set the corresponding parameters.* **initialize_method**=`OT`: using optimal transport. Key parameters:             * **OT_epsilon** (`float`; default: 0.02): the entropic regularization         * **OT_cost** ({`GED`, `SPD`}, default: `SPD`): method for computing the cost function. `GED` uses simple gene expression distances; it is faster. `SPD` uses the shortest path distances; it is slower but often more accurate. * **initialize_method**=`HighVar`: `HighVar` assumes that cells similar in gene expression across time points share clonal origin. It converts highly variable genes into pseudo multi-time clones and runs ``cs.tmap.infer_Tmap_from_multitime_clones`` to construct the map. We find `HighVar` performs better than `OT`, especially when there are large differentiation effects over the observed time window, or batch effects.    * **HighVar_gene_pctl** (`int`; default: 85): percentile threshold to select highly variable genes. Range: [0,100]. A higher value selects more variable genes.             * **max_iter_N** (`list`; default: [3,5]): a list of two entries, representing the maximum iteration number for joint optimization and CoSpar core function, respectively.     * **epsilon_converge** (`list`; default: [0.05,0.05]), a list of two entries, indicating the map convergence threshold for joint optimization and CoSpar core function, respectively.  Infer transition map from a single clonal time point (using `OT`), please run ( ~ 1 min):<jupyter_code>adata_1=cs.tmap.infer_Tmap_from_one_time_clones(adata_orig,initial_time_points=['4'],
        later_time_point='6',initialize_method='OT',OT_cost='GED',
                smooth_array=[20,15,10,5],sparsity_threshold=0.1)
cs.hf.check_available_map(adata_1)
adata_1.uns['available_map']<jupyter_output><empty_output><jupyter_text>Infer transition map from state information alone (using `HighVar`): (it takes ~ 2 mins)<jupyter_code>adata_2=cs.tmap.infer_Tmap_from_state_info_alone(adata_orig,initial_time_points=['4'],
        later_time_point='6',initialize_method='HighVar',HighVar_gene_pctl=85,max_iter_N=[10,10],
    epsilon_converge=[0.01,0.01],smooth_array=[20,15,10,5],sparsity_threshold=0.1)
cs.hf.check_available_map(adata_2)
adata_2.uns['available_map']<jupyter_output><empty_output><jupyter_text>Transition map from clonal data alone, without state information As a comparision, we can also learn the transition map using clonal information alone:<jupyter_code># We use the Weinreb method, which selects uni-potent clones based on a pre-defined list of fates.
selected_fates=['Baso', 'Ccr7_DC', 'Eos', 'Erythroid', 'Lymphoid', 'Mast', 'Meg', 'Monocyte', 'Neu_Mon', 'Neutrophil', 'pDC']

adata_3=cs.tmap.infer_Tmap_from_clonal_info_alone(adata_orig,method='weinreb',later_time_point='6',selected_fates=selected_fates)
cs.hf.check_available_map(adata_3)
adata_3.uns['available_map']<jupyter_output><empty_output><jupyter_text>Saving maps<jupyter_code>cs.hf.save_map(adata)<jupyter_output>Saved file: data_cospar/LARRY_sp500_ranking1_MultiTimeClone_Later_FullSpace0_t*2*4*6_adata_with_transition_map.h5ad
{'fraction_non_alphanumeric': 0.10013117621337997, 'fraction_numerical': 0.020988194140795804, 'mean_word_length': 6.223157894736842, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '523289', 'license': 'MIT', 'n_tokens_mistral': 2176, 'n_tokens_neox': 2025, 'n_words': 696, 'path': 'docs/source/20211010_map_inference.ipynb', 'repo_name': 'AllonKleinLab/cospar', 'size': '6861'}