<jupyter_start><jupyter_text>Annotate phylogeny of Californian SARS-CoV-2 sequences with mobility dataThis notebook is a proof of concept for how one might annotate phylogenies inferred from SARS-CoV-2 genomes with mobility data to better understand how changes in mobility are related to viral transmission.  SetupImport modules and define utility functions.<jupyter_code>from augur.utils import json_to_tree
import datetime
import json
import pandas as pd
import requests
# Convert numerical dates to ISO dates.
def float_to_datestring(time):
    """Convert a floating point date from TreeTime `numeric_date` to a date string
    """
    # Extract the year and remainder from the floating point date.
    year = int(time)
    remainder = time - year

    # Calculate the day of the year (out of 365 + 0.25 for leap years).
    tm_yday = int(remainder * 365.25)
    if tm_yday == 0:
        tm_yday = 1

    # Construct a date object from the year and day of the year.
    date = datetime.datetime.strptime("%s-%s" % (year, tm_yday), "%Y-%j")

    # Build the date string with zero-padded months and days.
    date_string = "%s-%.2i-%.2i" % (date.year, date.month, date.day)

    return date_string<jupyter_output><empty_output><jupyter_text>Prepare mobility data Download the Google Mobility data.<jupyter_code>mobility_data_url = "https://www.gstatic.com/covid19/mobility/Global_Mobility_Report.csv?cachebust=204bd65a761b3b2a"
df = pd.read_csv(mobility_data_url)<jupyter_output>/Users/jlhudd/miniconda3/envs/mobility/lib/python3.6/site-packages/IPython/core/interactiveshell.py:3072: DtypeWarning: Columns (4) have mixed types.Specify dtype option on import or set low_memory=False.
  interactivity=interactivity, compiler=compiler, result=result)<jupyter_text>Extract rows for California.<jupyter_code>california_df = df.query("country_region_code == 'US' & sub_region_1 == 'California'").copy()<jupyter_output><empty_output><jupyter_text>Extract rows with county-level data.<jupyter_code>county_level_df = california_df[~pd.isnull(california_df["sub_region_2"])].copy()<jupyter_output><empty_output><jupyter_text>Select a subset of relevant columns. In this notebook, we only use the change from baseline mobility for retail and recreation as an example.<jupyter_code>county_level_df = county_level_df.loc[:, ["sub_region_2", "date", "retail_and_recreation_percent_change_from_baseline"]].copy()<jupyter_output><empty_output><jupyter_text>Rename mobility column name for county to a more interpretable name.<jupyter_code>county_level_df["county"] = county_level_df["sub_region_2"].str.replace(" County", "")<jupyter_output><empty_output><jupyter_text>Prepare phylogenetic dataDownload the latest California tree from covidtracker.<jupyter_code>tree_url = "https://github.com/czbiohub/covidtracker/raw/master/auspice/covidtracker_ca.json"
response = requests.get(tree_url)<jupyter_output><empty_output><jupyter_text>Load the tree.<jupyter_code>tree_json = json.loads(response.content)
tree = json_to_tree(tree_json)<jupyter_output><empty_output><jupyter_text>Extract strain name, date, and county for each tip.<jupyter_code>attrs = [
    "county",
    "num_date"
]
tree_records = []
for tip in tree.find_clades(terminal=True):
    record = {"strain": tip.name}
    for attr in attrs:
        if attr in tip.node_attrs:
            record[attr] = tip.node_attrs[attr]["value"]
        else:
            record[attr] = "N/A"

    tree_records.append(record)
tree_df = pd.DataFrame(tree_records)<jupyter_output><empty_output><jupyter_text>Convert numerical dates to ISO dates.<jupyter_code>tree_df["date"] = tree_df["num_date"].apply(float_to_datestring)<jupyter_output><empty_output><jupyter_text>Annotate phylogenetic data with mobility data Annotate strain metadata with mobility data for the corresponding date and county. Drop records where we don't have any mobility information. Keep only a subset of the final columns.<jupyter_code>tree_with_mobility_df = tree_df.merge(
    county_level_df,
    on=["date", "county"],
    how="left"
).dropna().loc[:, ["strain", "date", "retail_and_recreation_percent_change_from_baseline"]].copy()<jupyter_output><empty_output><jupyter_text>Bin mobility data into quartiles for categorical view in Nextstrain. We have limited control over how colors appear for drag-and-drop metadata. For production use, we might prefer to color by the continuous mobility values and let Nextstrain provide an appropriate color ramp.Lower binned values correspond to reduced mobility.<jupyter_code>tree_with_mobility_df["binned_mobility"] = pd.qcut(
    tree_with_mobility_df["retail_and_recreation_percent_change_from_baseline"],
    4,
    labels=["0-25%", "25-50%", "50-75%", "75-100%"]
)<jupyter_output><empty_output><jupyter_text>Export annotated phylogenetic data for visualization in Nextstrain Save the annotated strain data to a CSV that we can drag-and-drop onto the tree in Nextstrain.<jupyter_code>tree_with_mobility_df.to_csv(
    "california_mobility.csv",
    header=True,
    index=False
)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09954572387912305, 'fraction_numerical': 0.011455658700375272, 'mean_word_length': 5.936986301369863, 'pattern_counts': {'":': 1, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '354778', 'license': 'MIT', 'n_tokens_mistral': 1696, 'n_tokens_neox': 1612, 'n_words': 408, 'path': '2020-09-03-annotate-sars-cov-2-phylogeny-with-mobility-data.ipynb', 'repo_name': 'huddlej/mobility-on-phylogenies', 'size': '5063'}