<jupyter_start><jupyter_text>Theoretical Efficiency of Read Until EnrichmentThe "Read Until" feature of the Oxford Nanopore sequencing technology means a program can see the data coming in at each pore and, dependend on that data, reject the molecule inside a certain pore.The actual performance of such a method depends on a lot of factors:* ratio of desireable over undesireable molecules in the sample* accuracy of detection* length of event data necessary for the decision* latency of event data reaching the controlling program* delay between decision and ejecting the molecule* time until the pore can accept a new molecule* length of DNA strands in the sampleIn this notebook I boiled it down to three parameters:* `ham_frequency` is the frequency of desired molecules* `ham_duration` is the scale by which the desired molecules are read "longer"* `accuracy` is the accuracy of the classificationThe analogy to spam detection is chosen because "ham/spam" makes for catchier variable names. This computation considers time and "amount of data" as equivalent. In reality, event speeds vary a lot, but in the long run, duration of reads and length of the strands correlate very strongly.The result of the computation is the ratio of desired time/data over undesired time/data, which is hopefully higher than the original `ham_frequency`.<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
def sim_ru(ham_frequency, ham_duration, accuracy):
    # Monte-Carlo Style
    n = 1000000
    ham = np.random.random(size=n)<ham_frequency
    durations = np.ones(n)
    accurate = np.random.random(size=n)<accuracy
    durations[ham & accurate] = ham_duration
    durations[~ham & ~accurate] = ham_duration
    return (np.sum(durations[ham]) / np.sum(durations))


def sim_ru2(ham_frequency, ham_duration, accuracy):
    # exact calculation
    long = ((ham_frequency* accuracy) + (1-ham_frequency)*(1-accuracy)) * ham_duration
    short = ((ham_frequency* (1-accuracy)) + (1-ham_frequency)*(accuracy)) * 1.0
    ham = (ham_frequency* accuracy)*ham_duration + (ham_frequency* (1-accuracy))*1
    return ham / (long+short)
def make_plot(ham_frequency):
    f, ax = plt.subplots()
    f.set_figwidth(14)
    f.set_figheight(6)
    ax.set_ylim(0,1)
    x = np.arange(0.5, 1.0,0.001)
    y = np.zeros(len(x))
    handles = []
    for j in reversed([2.5,5,10,20,40]):
        for i in range(len(x)):
            y[i] = sim_ru2(ham_frequency, j, x[i])
        handles.append(ax.plot(x,y, label = "%.1f" % j))
    ax.grid()
    f.suptitle("Ratio of desired data over total data for different values of \"desired length\"/\"rejected length\"  ")
    ax.legend(loc=0);
    ax.xaxis.set_label_text("Detection Accuracy");
    ax.yaxis.set_label_text("Desired Output /  Total Output");<jupyter_output><empty_output><jupyter_text>50% ham in sample<jupyter_code>make_plot(0.5)<jupyter_output><empty_output><jupyter_text>10% ham in sample<jupyter_code>make_plot(0.1)<jupyter_output><empty_output><jupyter_text>1% ham in sample<jupyter_code>make_plot(0.01)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09819121447028424, 'fraction_numerical': 0.017118863049095608, 'mean_word_length': 4.724584103512015, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '352947', 'license': 'ISC', 'n_tokens_mistral': 999, 'n_tokens_neox': 949, 'n_words': 321, 'path': 'examples/squiggle_classifier_1/Read_Until_Efficiency.ipynb', 'repo_name': 'adelq/porekit-python', 'size': '3096'}