<jupyter_start><jupyter_text>Evaluate running time by number of samples In this tutorial we evaluate the running time of ternadecov for a varying number of samples using the GP trajectory model. The configuration of this tutorial has been set so that it completes in under 5 minutes. In order to generate more interesting plots increase the `n_iters` to 20k and set `n_sample_range` to span the range 500-1000.<jupyter_code># load libraries
import anndata
import matplotlib.pylab as plt
import torch
from ternadecov.time_deconv import *
from ternadecov.simulator import *
from ternadecov.stats_helpers import *
from ternadecov.dataset import *
from ternadecov.deconvolution_plotter import *
from ternadecov.parametrization import *
from ternadecov.evaluation import evaluate_with_trajectory
from ternadecov.deconvolution_writer import DeconvolutionWriter
from ternadecov.deconvolution_exporter import DeconvolutionExporter<jupyter_output><empty_output><jupyter_text>Configure<jupyter_code># number of iterations to run
n_iters = 5_000

# path to single-cell anndata dataset
sc_anndata_path = "/home/nbarkas/disk2/deconvolution_method/datasets/ebov/load_data_python/ebov_sc.h5ad"
with open(sc_anndata_path, 'rb') as fh:
    sc_anndata = anndata.read_h5ad(fh)<jupyter_output><empty_output><jupyter_text>Simulate data<jupyter_code>types = DeconvolutionDatatypeParametrization()<jupyter_output><empty_output><jupyter_text>Next we generate a 'SingleCellDataset' object. This is a special type of deconvolution dataset object that only contains single-cell data and is used only for simulations.<jupyter_code>ebov_sc_dataset = SingleCellDataset(
    sc_anndata, 
    sc_celltype_col="Subclustering_reduced", 
    dtype_np=types.dtype_np, dtype = types.dtype, device=types.device)<jupyter_output><empty_output><jupyter_text>Sample a random trajectory of celltypes that will be used for all iterations by only varying the number of samples draw from it<jupyter_code>trajectory_coef = sample_trajectories(
    type = 'periodic',
    num_cell_types = ebov_sc_dataset.w_hat_gc.shape[1]
)
n_sample_range = list(range(10,90,20))+list(range(100,1000,200)) 
print(n_sample_range)
n_samples = []
l1_error_norm = []
fit_time = []


for n in n_sample_range:
    
    eval_result = evaluate_with_trajectory(
        sc_dataset = ebov_sc_dataset, 
        n_samples = n, 
        trajectory_type = 'periodic', 
        trajectory_coef = trajectory_coef,
        n_iters = n_iters,
        types = DeconvolutionDatatypeParametrization(),
        deconvolution_params = {
            'trajectory_model_type': 'gp',
            'hyperparameters': TimeRegularizedDeconvolutionModelParametrization(),
            'trajectory_hyperparameters': TimeRegularizedDeconvolutionGPParametrization(),
        }
    )
    
    n_samples.append(eval_result['n_samples'])
    l1_error_norm.append(eval_result['l1_error_norm'])
    fit_time.append(eval_result['fit_time'])
evaluation_resuts_df = pd.DataFrame({'n_samples':n_samples, 'l1_error_norm': list(x.item() for x in l1_error_norm), 'fit_time': fit_time})
evaluation_resuts_df
evaluation_resuts_df.to_csv('runtime_evaluation_results.csv')<jupyter_output><empty_output><jupyter_text>Plot results<jupyter_code># Plot run-time
plt.figure(figsize=(10, 6))
plt.rcParams.update({'font.size': 22})
plt.plot(evaluation_resuts_df.n_samples.to_numpy(), evaluation_resuts_df.fit_time.to_numpy(), color='black')
plt.plot([10],[3474], color='red', markersize=10, marker='o')
plt.xlabel("Number of bulk samples")
plt.ylabel("Time (s)")
plt.title("ternadecov run time")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08995294768890119, 'fraction_numerical': 0.014392471630224191, 'mean_word_length': 6.086274509803921, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '655608', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1272, 'n_tokens_neox': 1187, 'n_words': 256, 'path': 'notebooks/tutorials/tutorial-evaluate-run-time-gp.ipynb', 'repo_name': 'broadinstitute/temporal-rna-seq-deconvolution', 'size': '3613'}