<jupyter_start><jupyter_text>The `utils` PackageAs the name says, this package brings some extra functionalities that you might need while using Maybrain.Let's start by importing it and initialising a Brain:<jupyter_code>from maybrain import utils
from maybrain import resources as rr
from maybrain import brain as mbt

a = mbt.Brain()
a.import_adj_file(rr.DUMMY_ADJ_FILE_500)
a.import_spatial_info(rr.MNI_SPACE_COORDINATES_500)
a.apply_threshold()<jupyter_output><empty_output><jupyter_text>Information about PercentagesImagine that you want to know what would be the ratio between the edges on `adjMat` above a certain threshold value and the total possible edges of `adjMat` (the ones different from `nan`). This might be useful for you to decide which threshold you might apply later.In our specific matrix, we can verify that we have 124750 possible edges in `adjMat`, and if we applied a threshold of 0.6, we would get 3387 edges:<jupyter_code>print("Ratio:", utils.threshold_to_percentage(a, 0.6))

## Checking the previous result
# Creating all possible edges
a.apply_threshold() 
print("Total possible edges: ", a.G.number_of_edges())
# Getting the edges thresholded with 0.6
a.apply_threshold(threshold_type="tVal", value=0.6)
print("Number of edges from a threshold of 0.6: ", a.G.number_of_edges())

print("(3387/124750 = ", 3387/124750, ")")<jupyter_output>Ratio: 0.027150300601202406
Total possible edges:  124750
Number of edges from a threshold of 0.6:  3387
(3387/124750 =  0.027150300601202406 )<jupyter_text>While `threshold_to_percentage()` is based on values from `adjMat`, we also have another method to calculate a similar ratio from values of the `G` object. This method is `percent_connected()`, and it returns the ratio of the current number of edges in our `G` object and the total number of possible connections. You can see this difference with other aspects. For example, if `adjMat` has *NaN*s, they are not counted in the result of `threshold_to_percentage()`. On the other hand, `percent_connected()` calculates the number of total possible connections, using the following formula for an unidirected graph:$$\left (nodes \times \left (nodes - 1  \right )  \right ) / 2$$This is equivalent to the upper right side of an adjacency matrix, including possible *NaN*s that it might have. For a directed graph, the formula is:$$nodes \times \left (nodes - 1  \right )  $$<jupyter_code>a.apply_threshold()
print("Ratio: ", utils.percent_connected(a))<jupyter_output>Ratio:  1.0<jupyter_text>The previous ratio is equals to 1 because we applied a threshold where we included all the possible edges from `adjMat`, thus everything is connected.We can reach the same ratio value we showed before with `threshold_to_percentage()` if we apply a threshold of 0.6:<jupyter_code>a.apply_threshold(threshold_type="tVal", value=0.6)

print("Ratio in a.G after thresholding of 0.6: ", utils.percent_connected(a))

## Checking the previous value
nodes = a.G.number_of_nodes()
print("Total possible edges from G: ", (nodes * (nodes-1)) / 2)
print("Number of edges in G: ", a.G.number_of_edges())

print("(3387/124750 = ", 3387/124750, ")")<jupyter_output>Ratio in a.G after thresholding of 0.6:  0.027150300601202406
Total possible edges from G:  124750.0
Number of edges in G:  3387
(3387/124750 =  0.027150300601202406 )<jupyter_text>Highlighting BrainsProperties of a Brain can be filtered using highlights. A Highlight is simply a list of nodes and/or edges.In order to be easier to see the highlighting features, we will be importing a shorter matrix with just 4 nodes ([link here](https://github.com/RittmanResearch/maybrain/blob/master/docs/data/3d_grid_adj.txt)) and properties about colours which we already used before ([link here](https://github.com/RittmanResearch/maybrain/blob/master/docs/data/3d_grid_properties.txt)).<jupyter_code>from maybrain import constants as ct

b = mbt.Brain()
b.import_adj_file("data/3d_grid_adj.txt")
b.apply_threshold()
b.import_properties("data/3d_grid_properties.txt")

# Checking the properties we have for the nodes and edges, to confirm the next function calls
for edge in b.G.edges(data=True):
    print(edge)
for node in b.G.nodes(data=True):
    print(node)<jupyter_output>Warning! Unable to process property ['colour', 6, 'green']
(0, 1, {'colour': 'grey', 'weight': 0.60080034391699999})
(0, 2, {'colour': 'red', 'weight': 0.203602458588})
(0, 3, {'colour': 'green', 'weight': 0.16390494700200001})
(1, 2, {'colour': 'green', 'weight': 0.84379894778099995})
(1, 3, {'colour': 'green', 'weight': 0.242747996199})
(2, 3, {'colour': 'green', 'weight': 0.63727884848299998})
(0, {'colour': 'blue'})
(1, {'colour': 'red'})
(2, {})
(3, {'colour': 'red'})<jupyter_text>The main function to create the highlights is `highlight_from_conds(brain, prop, rel, val, mode, label)`. This function creates an highlight by asking if the property `prop` is related to `val` by `rel`.The highlight is then stored in a dictionary `utils.highlights`, where the keys are the ones passed previously in the parameter `label`. If you don't define a label, it will create one automatically.The `rel` parameter can be:```'geq' - greater than or equal to'leq' - less than or equal to'gt' - strictly greater than'lt' - strictly less than'eq' - equal to (i.e. exactly)'in()', 'in[)', 'in(]', 'in[]' - within an interval, in this case `val` is a list of two numbers                                "[" and "]" means inclusive, "(" and ")" means exclusive'in' - in `val````In the next example you can see this function in action. In the first case, we filter the brain by getting the nodes which have the property `colour` equals to `green`, and we save in the `highlights` dictionary with the label `1`. The second case is the same filtering, but applied to the edges, and saved in the `highlights` dictionary with the label `2`.<jupyter_code>utils.highlight_from_conds(b, 'colour', 'eq', 'green', mode='node', label=1)
# Getting the highlight with the label 1
highlight = utils.highlights[1]
# Printing the results of the highlight
print(highlight.edges) # Empty because we chose to highligh just the nodes
print(highlight.nodes) # Empty because there is no node with the property `colour` as `green`


utils.highlight_from_conds(b, 'colour', 'eq', 'green', mode='edge', label=2)
# Getting the highlight with the label 1
highlight = utils.highlights[2]
# Printing the results of the highlight
print(highlight.edges) # We have edges with the property `colour` as `green`
print(highlight.nodes) # Empty again<jupyter_output>[]
[]
[(0, 3), (1, 2), (1, 3), (2, 3)]
[]<jupyter_text>In the next example we use the relation `in` in different ways. In the first case we want to find the edges whose weights are between 0.6 and 0.8 (exclusive). In the second case we show that it is possible to highlight at the same time nodes and edges, by finding the edges/nodes whose `colour` property is `red` or `grey`.<jupyter_code>utils.highlight_from_conds(b, ct.WEIGHT, 'in()', (0.6, 0.8), mode='edge', label=3)
utils.highlight_from_conds(b, 'colour', 'in', ['red','grey'], mode='node|edge',  label=4)

# Getting the highlights and printing them
high3 = utils.highlights[3]
high4 = utils.highlights[4]
print(high3.edges)
print(high3.nodes)
#
print(high4.edges)
print(high4.nodes)<jupyter_output>[(0, 1), (2, 3)]
[]
[(0, 1), (0, 2)]
[1, 3]<jupyter_text>Just a last detail about this function. If you put the property `x`/`X`/`y`/`Y`/`z`/`Z`, it will look out for the respective value from the property `ct.XYZ` in the nodes.Finally, if you already have your set of nodes and/or edges as your own highlights, you can just store them using `make_highlight()`:<jupyter_code>utils.make_highlight(edge_inds=[(1,2), (2,3)], nodes_inds=[3], label='custom1')
print(utils.highlights['custom1'].nodes)
print(utils.highlights['custom1'].edges)<jupyter_output>[3]
[(1, 2), (2, 3)]
{'fraction_non_alphanumeric': 0.1199442119944212, 'fraction_numerical': 0.04792696842906048, 'mean_word_length': 5.3561643835616435, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '422090', 'license': 'Apache-2.0', 'n_tokens_mistral': 2743, 'n_tokens_neox': 2475, 'n_words': 954, 'path': 'docs/02 - The utils Package.ipynb', 'repo_name': 'RittmanResearch/maybrain', 'size': '7887'}