<jupyter_start><jupyter_text>inspectionTo test whether the predictions of `TweetyNet` models recovered key behavioral findings, we studied branch points in Bengalese finch song, where a syllable (e.g. 'b') can be followed by more than one type (e.g. 'b-b' and 'b-c' are two transitions that might occur).    It has been previously reported that the transition probabilities at these branch points are stable over days, but can be flexibly modified depending on behavioral context.  So that we could determine whether `TweetyNet` predictions would allow us to reproduce this finding, we first visually inspected plots of transition matrices to look for branch points.Then, for each of the four birds in the Bengalese Finch song repository, we identified a branch point that we could verify reproduced the previously reported finding of remaining stable across days, as shown with permutation tests.Plots of the transition matrices and tests of the stability of the branch point probabilities that we identified are shown below. 1. Load annotations from dataset .csv files generated by running `vak prep`<jupyter_code>behav_configs_root = pyprojroot.here() / 'data/configs/Bengalese_Finches/behavior'
animal_ID_roots = sorted(
    [subdir
     for subdir in behav_configs_root.iterdir()
     if subdir.is_dir()]
)
animal_day_config_map = {}
for animal_ID_root in animal_ID_roots:
    animal_day_config_map[animal_ID_root.name] = sorted([subdir
                                                         for subdir in animal_ID_root.iterdir()
                                                         if subdir.is_dir()])
animal_day_config_map = {
    animal_ID: {
        day_dir.name: sorted(day_dir.glob('*.toml'))[0] for day_dir in day_dirs
    }
    for animal_ID, day_dirs in animal_day_config_map.items()
    
}
animal_day_transmats = {}
for animal_id, day_config_map in animal_day_config_map.items():
    day_transmats = {}
    for day, config_path in day_config_map.items():
        with config_path.open('r') as fp:
            config = toml.load(fp)
        prep_csv_path = pyprojroot.here() / config['EVAL']['csv_path']
        df = pd.read_csv(prep_csv_path)
        annots = vak.annotation.from_df(df)
        day_transmats[day] = article.bfbehav.sequence.transmat_from_annots(annots, thresh=0.002)
    animal_day_transmats[animal_id] = day_transmats<jupyter_output><empty_output><jupyter_text>plot transition matrices for each day, find a branch point whose probability remains constant<jupyter_code>article.plot.trans_mat.trans_mat_all_days(animal_day_transmats, animal_id='bl26lb16')
p_vals, alpha = article.bfbehav.stats.perm_test_across_days(animal_day_transmats,
                                                            animal_id='bl26lb16',
                                                            from_state='b',
                                                            transition=('b', 'c'),
                                                            n_perm=1000,)
article.plot.trans_mat.trans_mat_all_days(animal_day_transmats, animal_id='gy6or6', figsize=(15, 5))
p_vals, alpha = article.bfbehav.stats.perm_test_across_days(animal_day_transmats,
                                                            animal_id='gy6or6',
                                                            from_state='e',
                                                            transition=('e', 'f'),
                                                            n_perm=1000,)
article.plot.trans_mat.trans_mat_all_days(animal_day_transmats, animal_id='or60yw70', figsize=(15, 5))
p_vals, alpha = article.bfbehav.stats.perm_test_across_days(animal_day_transmats,
                                                            animal_id='or60yw70',
                                                            from_state='d',
                                                            transition=('d', 'e'),
                                                            n_perm=1000,)
article.plot.trans_mat.trans_mat_all_days(animal_day_transmats, animal_id='gr41rd51', figsize=(15, 5))
p_vals, alpha = article.bfbehav.stats.perm_test_across_days(animal_day_transmats,
                                                            animal_id='gr41rd51',
                                                            from_state='e',
                                                            transition=('e', 'f'),
                                                            n_perm=1000,)<jupyter_output>performing permutation test for: 2012-06-20, 2012-06-22
p-value was 0.183
performing permutation test for: 2012-06-20, 2012-06-23
p-value was 0.196
performing permutation test for: 2012-06-22, 2012-06-20
p-value was 0.167
performing permutation test for: 2012-06-22, 2012-06-23
p-value was 0.886
performing permutation test for: 2012-06-23, 2012-06-20
p-value was 0.208
performing permutation test for: 2012-06-23, 2012-06-22
p-value was 0.905
none of the p-values were less than corrected alpha 0.008333333333333333.
Fail to reject the null hypothesis. p-values: [0.183, 0.196, 0.167, 0.886, 0.208, 0.905]
{'fraction_non_alphanumeric': 0.07888040712468193, 'fraction_numerical': 0.04345273047563124, 'mean_word_length': 2.0254588513913556, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 36, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '52586', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1550, 'n_tokens_neox': 1350, 'n_words': 366, 'path': 'article/src/scripts/Bengalese_Finches/behavior/branch-point-inspection.ipynb', 'repo_name': 'yardencsGitHub/tweetynet', 'size': '5109'}