<jupyter_start><jupyter_text>**Extract subtree for neuronal cells only**`Meis2 Adamts19` and `CR Lhx5` are outlier types that are ignored by choosing the root node to be `n4`<jupyter_code>#subtree is a instance of class HTree as well: 
subtree = htree.get_subtree(node='n4')
subtree.plot((18,8),fontsize=12)
#Work with a smaller tree for the examples below:
subtree = htree.get_subtree(node='n71')
subtree.plot((6,3),fontsize=12)<jupyter_output><empty_output><jupyter_text>**Reduce resolution of the hierarchical tree** - `get_mergeseq` returns list of merges to carry out, sorted by the depth. - This is equivalent to starting at the bottom of the tree and making successive horizontal cuts to reduce the number of 'leaf' nodes<jupyter_code>L = subtree.get_mergeseq()

for l in L:
    print(l)<jupyter_output>[['Vip Ptprt Pkp2', 'Vip Rspo4 Rxfp1 Chat'], 'n74']
[['Vip Lect1 Oxtr', 'Vip Rspo1 Itga4'], 'n75']
[['Vip Chat Htr1f', 'Vip Pygm C1ql1'], 'n72']
[['n74', 'n75'], 'n73']
[['n72', 'n73'], 'n71']<jupyter_text>**Apply changes to the hierarchy to a list of sample labels** - `sample_labels` is a `numpy array` of cells labels assigned based on the hierarchical clustering.  - In the example below, we set `sample_labels` for 6 cells by hand.<jupyter_code>from analysis_tree_helpers import do_merges
sample_labels = np.array(['Vip Ptprt Pkp2', 
                          'Vip Ptprt Pkp2',
                          'Vip Rspo4 Rxfp1 Chat',
                          'Vip Chat Htr1f', 
                          'Vip Pygm C1ql1',
                          'Vip Pygm C1ql1'])

#Perform successive merges of the hierarchy:
#If the list of labels does not contain any label that is merged in the i-th step, nothing changes.

for n in range(len(L)):
    print('\n--- n merges = {:d} ---'.format(n))
    merged_sample_labels = do_merges(labels=sample_labels, list_changes=L, n_merges=n, verbose=True)
    
    print('\nUpdated labels::'.format(n))
    for l in merged_sample_labels:
        print(l)<jupyter_output>--- n merges = 0 ---

Updated labels::
Vip Ptprt Pkp2
Vip Ptprt Pkp2
Vip Rspo4 Rxfp1 Chat
Vip Chat Htr1f
Vip Pygm C1ql1
Vip Pygm C1ql1

--- n merges = 1 ---
2  in  Vip Ptprt Pkp2  -->  n74
1  in  Vip Rspo4 Rxfp1 Chat  -->  n74

Updated labels::
n74
n74
n74
Vip Chat Htr1f
Vip Pygm C1ql1
Vip Pygm C1ql1

--- n merges = 2 ---
0  in  Vip Ptprt Pkp2  -->  n74
0  in  Vip Rspo4 Rxfp1 Chat  -->  n74
0  in  Vip Lect1 Oxtr  -->  n75
0  in  Vip Rspo1 Itga4  -->  n75

Updated labels::
n74
n74
n74
Vip Chat Htr1f
Vip Pygm C1ql1
Vip Pygm C1ql1

--- n merges = 3 ---
0  in  Vip Ptprt Pkp2  -->  n74
0  in  Vip Rspo4 Rxfp1 Chat  -->  n74
0  in  Vip Lect1 Oxtr  -->  n75
0  in  Vip Rspo1 Itga4  -->  n75
1  in  Vip Chat Htr1f  -->  n72
2  in  Vip Pygm C1ql1  -->  n72

Updated labels::
n74
n74
n74
n72
n72
n72

--- n merges = 4 ---
0  in  Vip Ptprt Pkp2  -->  n74
0  in  Vip Rspo4 Rxfp1 Chat  -->  n74
0  in  Vip Lect1 Oxtr  -->  n75
0  in  Vip Rspo1 Itga4  -->  n75
0  in  Vip Chat Htr1f  -->  n72
0  in  Vip Py[...]<jupyter_text>**Plot a custom subset as leaf nodes**<jupyter_code>subtree = htree.get_subtree(node='n71')
subtree.plot((6,3),fontsize=12)
fig_tree = plt.gcf()

#Choose nodes to be leaf nodes in the partial tree
kept_leaf_nodes = ['n74','n75','Vip Chat Htr1f','Vip Pygm C1ql1']

#Obtain all relevant ancestor nodes:
kept_tree_nodes=[]
for node in kept_leaf_nodes:
    kept_tree_nodes.extend(subtree.get_ancestors(node))
    kept_tree_nodes.extend([node])

kept_subtree_df = subtree.obj2df()
kept_subtree_df = kept_subtree_df[kept_subtree_df['child'].isin(kept_tree_nodes)]

#Plot updated tree:
print('\n\n---- Plot: New leaf subset: ----')
kept_subtree = HTree(htree_df=kept_subtree_df)
kept_subtree.plot((6,3),fontsize=12,txtleafonly=True,skeletoncol='#444444',fig=fig_tree)
plt.show()

print('\n\n---- Plot: Leaf nodes pushed to 0: ----')
kept_subtree_df['isleaf'].loc[kept_subtree_df['child'].isin(kept_leaf_nodes)]=True
kept_subtree_df['y'].loc[kept_subtree_df['child'].isin(kept_leaf_nodes)]=0.0
mod_subtree = HTree(htree_df=kept_subtree_df)
mod_subtree.plot((6,3),fontsize=12,txtleafonly=True,skeletoncol='#444444',fig=None)
plt.show()

print('\n\n---- Plot: Leaf nodes evenly distributed: ----')
mod_subtree.update_layout()
mod_subtree.plot((6,3),fontsize=12,txtleafonly=True,skeletoncol='#444444',fig=None)
plt.show()<jupyter_output>---- Plot: New leaf subset: ----<jupyter_text>**Examples demonstrating additional functions**<jupyter_code>#Extract an inhibitory subtree from this tree. subtree is a instance of class HTree as well:
subtree = htree.get_subtree(node='n61')
subtree.plot((6,3),fontsize=12)
#Get all ancestor nodes as a list
subtree.get_ancestors(node='n72')
#Get all ancestor nodes as a list, upto a specified rootnode
subtree.get_ancestors(node='n72',rootnode='n63')
#Get all descendants of a given node
subtree.get_descendants(node='n64')
#Get all descendants of a given node, excluding non-leaf nodes
subtree.get_descendants(node='n64',leafonly=True)
#Get sequence of merges that respects the hierarchy:
L = subtree.get_mergeseq()
for l in L:
    print(l)<jupyter_output>[['Vip Arhgap36 Hmcn1', 'Vip Gpc3 Slc18a3'], 'n70']
[['Vip Ptprt Pkp2', 'Vip Rspo4 Rxfp1 Chat'], 'n74']
[['Vip Crispld2 Htr2c', 'Vip Crispld2 Kcne4'], 'n65']
[['Vip Lect1 Oxtr', 'Vip Rspo1 Itga4'], 'n75']
[['Vip Chat Htr1f', 'Vip Pygm C1ql1'], 'n72']
[['Vip Igfbp6 Car10', 'Vip Igfbp6 Pltp'], 'n62']
[['n74', 'n75'], 'n73']
[['Vip Lmo1 Fam159b', 'Vip Lmo1 Myl1'], 'n67']
[['Vip Igfbp4 Mab21l1', 'n70'], 'n69']
[['Vip Col15a1 Pde1a', 'n65'], 'n64']
[['n72', 'n73'], 'n71']
[['n69', 'n71'], 'n68']
[['n67', 'n68'], 'n66']
[['n64', 'n66'], 'n63']
[['n62', 'n63'], 'n61']<jupyter_text>**Valid partitions** - Many different partitions that consistent with the topology but not depths of the nodes in the hierarchical tree.  - Below we consider a small hierarchical tree, and list all such partitions.<jupyter_code>from analysis_tree_helpers import Node, get_valid_classifications

#Define a subtree:
subtree = htree.get_subtree(node='n73')
subtree.plot(figsize=(4,2.5),fontsize=12)

#Get a pandas dataframe for this tree:
subtree_df=subtree.obj2df()
print(subtree_df)
#Define the root node to calculate the different classifications:
start_node = Node(name='n73',
                  C_list=subtree_df['child'].values,
                  P_list=subtree_df['parent'].values)

#Get all possible classifications consistent with the tree defined by subtree_df
valid_classifications = []
valid_classifications = get_valid_classifications([start_node],
                                                  C_list=subtree_df['child'].values,
                                                  P_list=subtree_df['parent'].values,
                                                  valid_classes=valid_classifications)

#List all the classifications, sorted by the number of classes that remain
valid_classifications.sort(key=len)
for v in valid_classifications:
    print(v)<jupyter_output>['n73']
['n74', 'n75']
['Vip Ptprt Pkp2', 'Vip Rspo4 Rxfp1 Chat', 'n75']
['Vip Lect1 Oxtr', 'Vip Rspo1 Itga4', 'n74']
['Vip Lect1 Oxtr', 'Vip Ptprt Pkp2', 'Vip Rspo1 Itga4', 'Vip Rspo4 Rxfp1 Chat']<jupyter_text>**Calculating number of partitions for the major inhibitory cell classes**<jupyter_code>#Load a tree from the .csv file
htree = HTree(htree_file='./tree_20180520.csv')
subtree = htree.get_subtree(node='n59')

subclass_names = ['Lamp+Sncg','Vip','Sst','Pvalb']
for k,n in enumerate(['n76','n61','n90','n109']):
    subtree = htree.get_subtree(node=n)
    subtree_df=subtree.obj2df()
    start_node = Node(name=n,C_list=subtree_df['child'].values,P_list=subtree_df['parent'].values)
    valid_classifications = []
    valid_classifications = get_valid_classifications([start_node],
                                                      C_list=subtree_df['child'].values,
                                                      P_list=subtree_df['parent'].values,
                                                      valid_classes=valid_classifications)
    print('{} can be partitioned in {} ways'.format(subclass_names[k],len(valid_classifications)))
%matplotlib inline
from analysis_tree_helpers import HTree,do_merges

#Load a tree from the .csv file
htree = HTree(htree_file='./tree_20180520.csv')

#subtree is a instance of class HTree as well: 
subtree = htree.get_subtree(node='n3')
L = subtree.get_mergeseq()
merged_sample_labels = do_merges(labels=subtree.child[subtree.isleaf], list_changes=L, n_merges=30)
kept_leaf_nodes = np.unique(merged_sample_labels).tolist()
#Choose nodes to be leaf nodes in the partial tree
#kept_leaf_nodes = ['n74','n75','Vip Chat Htr1f','Vip Pygm C1ql1']

#Obtain all relevant ancestor nodes:
kept_tree_nodes=[]
for node in kept_leaf_nodes:
    kept_tree_nodes.extend(subtree.get_ancestors(node))
    kept_tree_nodes.extend([node])

kept_subtree_df = subtree.obj2df()
kept_subtree_df = kept_subtree_df[kept_subtree_df['child'].isin(kept_tree_nodes)]

#Plot updated tree:
print('\n\n---- Plot: New leaf subset: ----')
kept_subtree = HTree(htree_df=kept_subtree_df)
kept_subtree.plot((18,8),fontsize=12,txtleafonly=False,skeletoncol='#444444',fig=None)
plt.show()

print('\n\n---- Plot: Leaf nodes pushed to 0: ----')
kept_subtree_df['isleaf'].loc[kept_subtree_df['child'].isin(kept_leaf_nodes)]=True
kept_subtree_df['y'].loc[kept_subtree_df['child'].isin(kept_leaf_nodes)]=0.0
mod_subtree = HTree(htree_df=kept_subtree_df)
mod_subtree.plot((18,8),fontsize=12,txtleafonly=True,skeletoncol='#444444',fig=None)
plt.show()

print('\n\n---- Plot: Leaf nodes evenly distributed: ----')
mod_subtree.update_layout()
mod_subtree.plot((18,8),fontsize=12,txtleafonly=True,skeletoncol='#444444',fig=None)
plt.show()<jupyter_output>---- Plot: New leaf subset: ----
{'fraction_non_alphanumeric': 0.13796277357332787, 'fraction_numerical': 0.045510329310697484, 'mean_word_length': 4.119895287958115, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '210895', 'license': 'MIT', 'n_tokens_mistral': 3970, 'n_tokens_neox': 3709, 'n_words': 914, 'path': 'examples.ipynb', 'repo_name': 'AllenInstitute/celltype_hierarchy', 'size': '9778'}