<jupyter_start><jupyter_text>TTI Explorer `tti_explorer` is a library for simulating infection spread. This library is built to explore the impact of various test-trace-isolate strategies and social distancing measures on the spread of COVID-19 in the UK. This notebook is an introduction to the functionality offered by `tti-explorer`.<jupyter_code>%pwd
%cd ~/Desktop/College\ Work/Fourth\ Year/L48/L48Project/tti-explorer
%pip install -q .
import os

import numpy as np
import pandas as pd
from tqdm.notebook import trange
from tqdm import tqdm

from tti_explorer import config, utils
from tti_explorer.case import simulate_case, CaseFactors
from tti_explorer.contacts import EmpiricalContactsSimulator
from tti_explorer.strategies import TTIFlowModel, RETURN_KEYS
def print_doc(func):
    print(func.__doc__)<jupyter_output><empty_output><jupyter_text>Before we do anything, let's make a random state<jupyter_code>rng = np.random.RandomState(0)<jupyter_output><empty_output><jupyter_text>We will first do a short tour of the functionality, then show how this is put together to generate simulation results.  Generate a case The function we use for this is `simulate_case` in `case.py`<jupyter_code>print_doc(simulate_case)<jupyter_output>simulate_case

    Args:
        rng (np.random.RandomState): random number generator.
        p_under18 (float): Probability of case being under 18
        infection_proportions (dict): Probs of being symp covid neg, symp covid pos, asymp covid pos
                                      The only required key is 'dist', which contains list of the named probs, in that order.
        p_day_noticed_symptoms (np.array[float]): Distribution of day on which case notices
            their symptoms. (In our model this is same as reporting symptoms.)
            Conditional on being symptomatic.
        inf_profile (list[float]): Distribution of initial exposure of positive secondary cases
            relative to start of primary case's infectious period.

    Returns (Case): case with attributes populated.<jupyter_text>We store our config values in `config.py`. You can retrieve them as follows<jupyter_code>case_config = config.get_case_config("delve")
case_config<jupyter_output><empty_output><jupyter_text>We use these parameters to simulate a case<jupyter_code>primary_case = simulate_case(rng, **case_config)
print_doc(primary_case)<jupyter_output>Case(under18: bool, covid: bool, symptomatic: bool, day_noticed_symptoms: int, inf_profile: list)<jupyter_text>Returned is a `case.Case` with stochastically generated attributes.  Deeper: Case attributes Let's go through the simulated attributes of a `case.Case`. The attributes `.under18`, `.covid` and `.symptomatic` are `bool` types indicating whether the generated `case.Case` is under 18, COVID positive and symptomatic respectively. All possible values of these attributes are possible apart from the combination `.covid = False` and `.symptomatic = False` (a configuration irrelevant for the purpose of simulating infection spread). The primary case we just simulated has the following attributes:<jupyter_code>print(f'Under 18: {primary_case.under18}.')
print(f'COVID positive: {primary_case.covid}.')
print(f'Symptomatic: {primary_case.symptomatic}.')<jupyter_output>Under 18: False.
COVID positive: False.
Symptomatic: True.<jupyter_text>Each `case.Case` also has an attribute `.day_noticed_symptoms` of type `int`, indicating the number of days from start of infectiousness until the `case.Case` noticed the symptoms. If a `case.Case` is asymptomatic, the attribute `.day_noticed_symptoms` is set to `-1`.<jupyter_code>print(f'primary_case noticed symptoms {primary_case.day_noticed_symptoms} days after start of infectiousness.')<jupyter_output>primary_case noticed symptoms 3 days after start of infectiousness.<jupyter_text>Finally, the attribute `.inf_profile` is a `list` describing the relative infectiousness of the case for each day of the infectious period. If `.covid = False` for a `case.Case`, this is `0` throughout.<jupyter_code>print(f'inf_profile is: {primary_case.inf_profile}')<jupyter_output>inf_profile is: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]<jupyter_text>As mentioned above, the configuration for simulating these attributes are stored in `config.py`. This includes the distributions used for sampling attributes. For instance, the attribute `.under18` is sampled from a Bernoulli distribution with probability `0.21`:<jupyter_code>print(f'Probability of case being under 18: {case_config["p_under18"]}')<jupyter_output>Probability of case being under 18: 0.21<jupyter_text>As another example, if `case.Case` is symptomatic, the attribute `.days_noticed_symptoms` is sampled from a categorical distribution over the set {0, 1, ..., 9} (since we model an infection period of ten days in this configuration) with probabilities:<jupyter_code>print(f'Probability distribution of .day_noticed_symptoms: {case_config["p_day_noticed_symptoms"]}')<jupyter_output>Probability distribution of .day_noticed_symptoms: [0, 0.25, 0.25, 0.2, 0.1, 0.05, 0.05, 0.05, 0.05, 0.0]<jupyter_text>Generate contactsSocial contacts are represented by `Contacts` and defined in `contacts.py`.To simulate social contacts, we use the BBC Pandemic Dataset. This is stratified as over/under 18 to give different patterns of social contact depending on the age of the case.<jupyter_code>def load_csv(pth):
    return np.loadtxt(pth, dtype=int, skiprows=1, delimiter=",")
path_to_bbc_data = os.path.join("..", "data", "bbc-pandemic")

over18 = load_csv(os.path.join(path_to_bbc_data, "contact_distributions_o18.csv"))
under18 = load_csv(os.path.join(path_to_bbc_data, "contact_distributions_u18.csv"))<jupyter_output><empty_output><jupyter_text>Now that we have the data loaded, we use `EmpiricalContactsSimulator` to sample these tables for contacts of the primary case, then simulate their infection under a no measures scenario (i.e. no government intervention)<jupyter_code>print_doc(EmpiricalContactsSimulator.__init__)
simulate_contacts = EmpiricalContactsSimulator(over18, under18, rng)<jupyter_output><empty_output><jupyter_text>We can now use the callable `simulate_contacts` to simulate social contacts of the primary case<jupyter_code>print_doc(simulate_contacts.__call__)<jupyter_output>Generate a social contact for the given case.

        A row from the table corresponding to the age of the `case` is sampled
        uniformly at random. A contact is generated with daily contacts as
        given by that row. These contacts are infected at random with attack rates
        given by the SARs and whether or not the `case` is symptomatic. If the
        `case` is COVID negative, then no contacts are infected.

        Args:
            case (Case): Primary case.
            home_sar (float): Secondary attack rate for household contacts.
                              (Marginal probability of infection over the whole simulation)
            work_sar (float): Secondary attack rate for contacts in the work category.
            other_sar (float): Secondary attack rate for contacts in the other category.
            asymp_factor (float): Factor by which to multiply the probabilty of secondary
                                  infection if `case` is asymptomatic COVID positive[...]<jupyter_text>To do this we need some more parameters, which we also load from `config.py`. The user can, of course, specify this themselves if they would like.<jupyter_code>contacts_config = config.get_contacts_config("delve")
contacts_config.keys()<jupyter_output><empty_output><jupyter_text>We now do the same as we did with when simulating a primary case.<jupyter_code>social_contacts = simulate_contacts(primary_case, **contacts_config)
print_doc(social_contacts)<jupyter_output>Contacts(n_daily:dict, home:<built-in function array>, work:<built-in function array>, other:<built-in function array>)<jupyter_text>Deeper: Contacts attributes Let's examine the attributes of `social_contacts`, which is an instance of `contacts.Contacts`. Note that `social_contacts` is simulated above by calling `simulate_contacts` which takes `primary_case` as in argument, so contact generation of course depends on the case simulated first.  The first attribute to note is `.n_daily`, which is a `dict` containing the average number of daily contacts (split into three categories) of the case. This is simulated by sampling one row of the tables `over18` or `under18` depending on the value of `primary_case.under18`. In the case of `primary_case`, we can look at `social_contacts.n_daily`:<jupyter_code>print(f'Average number of daily contacts for primary_case:')
print(f'Home: {social_contacts.n_daily["home"]}')
print(f'Work: {social_contacts.n_daily["work"]}')
print(f'Other: {social_contacts.n_daily["other"]}')<jupyter_output>Average number of daily contacts for primary_case:
Home: 7
Work: 3
Other: 4<jupyter_text>The three remaining attributes `.home`, `.work` and `.other` are arrays containing information about each contact made by the case, with one row per contact. More specifically, for each contact, the row contains the first day (always measured relative to the start of infectiousness) of encounter between the case and contact and, if transmission occurred, then the day of transmission.  Also, recall that home contacts are assumed to repeat every day of the infectious period, whereas work/other contacts are new for each day. This means the lengths of the arrays `.work` and `.other` are `10 * .n_daily['work']` and `10 * .n_daily['other']` respectively (recalling the infection period is assumed to last ten days, a parameter set in `contacts_config['period']`). Whereas, the length of the `.home` array is just `.n_daily['home']`.<jupyter_code>print(f'Lengths of .home, .work and .other attributes:')
print(f'Home: {len(social_contacts.home)}')
print(f'Work: {len(social_contacts.work)}')
print(f'Other: {len(social_contacts.other)}')<jupyter_output>Lengths of .home, .work and .other attributes:
Home: 7
Work: 30
Other: 40<jupyter_text>Digging further into the array, each row contains two integers. The first integer indicates the day of transmission, which is set to `-1` if no transmission occurred. The second integer contains the day of first encounter. So for instance, looking at one of the home contacts, we see transmission didn't occur and the day of first encounter is `0`, i.e. the first day of the infection period:<jupyter_code>print(social_contacts.home[0])<jupyter_output>[-1  0]<jupyter_text>Looking at the first six work contacts, we see none of them were infected either. This is consistent with the fact that `primary_case.covid = False` so no transmission can occur in this case.<jupyter_code>print(social_contacts.work[:6])<jupyter_output>[[-1  0]
 [-1  0]
 [-1  0]
 [-1  1]
 [-1  1]
 [-1  1]]<jupyter_text>In simulations where `case.Case` is COVID positive, each contact may get infected and the probability of getting infected depends on parameters such as the secondary attack rates (SARs), all of which are set in `contacts_config`. For details on the precise simulation procedure used to generate `contacts.Contacts`, see either Appendix A of the report or the `__call__` method of `EmpiricalContactsSimulator`.  TTI Strategies All of the information about the primary case's infection and how they infect their social contacts (under no government intervention) is now contained in `primary_case` and `social_contacts`.Now we run a simulation, which works as follows. We start by generating a large number of cases, each with associated contacts. Given a particular strategy (e.g. test-based TTI with NPI of stringency level S3), each case is passed through the strategy, which computes various metrics for the case. For example, it computes the number of secondary cases due to primary case (reproduction number) and the number of tests required. We then collect the results for each case and average them, returning the final evaluation of the strategy.  Running a Simulation<jupyter_code>from tti_explorer.strategies import TTIFlowModel<jupyter_output><empty_output><jupyter_text>We will analyse the `S3_test_based_TTI` strategy from our report. For clarity, we will show the whole process. First get the configurations:<jupyter_code>name = 'S3_test_based_TTI'

case_config = config.get_case_config("delve")
print(case_config)

contacts_config = config.get_contacts_config("delve")
policy_config = config.get_strategy_configs("delve", name)[name]
factor_config = utils.get_sub_dictionary(policy_config, config.DELVE_CASE_FACTOR_KEYS)
strategy_config = utils.get_sub_dictionary(policy_config, config.DELVE_STRATEGY_FACTOR_KEYS)<jupyter_output>{'p_under18': 0.21, 'infection_proportions': {'dist': [0.8333333333333334, 0.1, 0.06666666666666667], 'nppl': 120}, 'p_day_noticed_symptoms': [0, 0.25, 0.25, 0.2, 0.1, 0.05, 0.05, 0.05, 0.05, 0.0], 'inf_profile': [0.046966101377360424, 0.15602255610429985, 0.19829974712514023, 0.18356485224565827, 0.14541407040442172, 0.10500447388376151, 0.07130993362939089, 0.04635772205156416, 0.029167894888682697, 0.017892648289720214]}<jupyter_text>Set a random state:<jupyter_code>rng = np.random.RandomState(42)<jupyter_output><empty_output><jupyter_text>Make contact simulator:<jupyter_code>simulate_contacts = EmpiricalContactsSimulator(over18, under18, rng)<jupyter_output><empty_output><jupyter_text>Make the TTI Model:<jupyter_code>tti_model = TTIFlowModel(rng, **strategy_config)<jupyter_output><empty_output><jupyter_text>Generate cases, contacts and run simulation:<jupyter_code>n_cases = 10000
outputs = list()

for i in tqdm(range(n_cases)):
    case = simulate_case(rng, **case_config)
    case_factors = CaseFactors.simulate_from(rng, case, **factor_config)
    contacts = simulate_contacts(case, **contacts_config)
    res = tti_model(case, contacts, case_factors)
    outputs.append(res)<jupyter_output>{'p_under18': 0.21, 'infection_proportions': {'dist': [0.8333333333333334, 0.1, 0.06666666666666667], 'nppl': 120}, 'p_day_noticed_symptoms': [0, 0.25, 0.25, 0.2, 0.1, 0.05, 0.05, 0.05, 0.05, 0.0], 'inf_profile': [0.046966101377360424, 0.15602255610429985, 0.19829974712514023, 0.18356485224565827, 0.14541407040442172, 0.10500447388376151, 0.07130993362939089, 0.04635772205156416, 0.029167894888682697, 0.017892648289720214]}
RandomState(MT19937)
k<jupyter_text>Collate and average results across the cases simulated:<jupyter_code># This cell is mosltly just formatting results...
to_show = [
    RETURN_KEYS.base_r,
    RETURN_KEYS.reduced_r,
    RETURN_KEYS.man_trace,
    RETURN_KEYS.app_trace,
    RETURN_KEYS.tests
]

# scale factor to turn simulation numbers into UK population numbers
nppl = case_config['infection_proportions']['nppl']
scales = [1, 1, nppl, nppl, nppl]

results = pd.DataFrame(
    outputs
).mean(
    0
).loc[
    to_show
].mul(
    scales
).to_frame(
    name=f"Simulation results: {name.replace('_', ' ')}"
).rename(
    index=lambda x: x + " (k per day)" if x.startswith("#") else x
)

results.round(1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08897742363877822, 'fraction_numerical': 0.04369189907038513, 'mean_word_length': 5.4890133563119345, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 93, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 4, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '14511', 'license': 'MIT', 'n_tokens_mistral': 5102, 'n_tokens_neox': 4564, 'n_words': 1585, 'path': 'tti/notebooks/tti-example-experiment.ipynb', 'repo_name': 'ashwinahuja/HowQuicklyCanWeGetBackToThePub', 'size': '15060'}