<jupyter_start><jupyter_text>**What is it like to be a squirrel monkey?** Analysis of open histological data using MicroDraw. HIBALL Winterschool workshop 2021<NAME>, <NAME>, <NAME> & <NAME> This collab demonstrates how to programmatically obtain all annotations available for whole-brain histological datasets in MicroDraw.The script also displays all annotations, and can synthesise them into a single histological volume. This allows, for example, to measure the volume of the different structures annotated.In this example, data comes from the [Squirrel monkey project](https://microdraw.pasteur.fr/project/squirrelmonkey).In this workshop, we will manually and collaboratively draw the pial surface of the cerebrum. Our aim will be to recover those annotations and produce a plot of the volume and surface for the cerebrum.  1 Manually and collaboratively draw the pial surface of the Squirrel Monkey brainEach of us will draw a slice, and we will then analyse our collaborative work to compute basic parameters (surface area and volume).We will use a framapad document (like Google doc but no need for account) to communicate and to tell each other which slices are we annotating.Here is the URL of our framapad document: https://mensuel.framapad.org/p/squirrelmonkeyDepending on whether you have a tablet with a pen, a mouse or a trackpad, this will take us from 3 to 15 minutes – and also depending on the slice you chose ;)  2 Download and install the microdraw packageWe will first install the `microdraw.py` module, which has several functions for accessing more easily data in the Microdraw website.We will only use the most basic functions of `microdraw.py` (which is very much work in progress, especially concerning best practices, documentation, and code organisation... 🙃).<jupyter_code>%%bash
rm -r microdraw
git clone https://github.com/r03ert0/microdraw.py
mv microdraw.py microdraw
pip install -r microdraw/requirements.txt<jupyter_output>Requirement already satisfied: bezier==2020.5.19 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line 1)) (2020.5.19)
Requirement already satisfied: certifi==2020.12.5 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line 2)) (2020.12.5)
Requirement already satisfied: cycler==0.10.0 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line 3)) (0.10.0)
Requirement already satisfied: decorator==4.4.2 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line 4)) (4.4.2)
Requirement already satisfied: distro==1.4.0 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line 5)) (1.4.0)
Requirement already satisfied: gitdb2==2.0.5 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line 6)) (2.0.5)
Requirement already satisfied: GitPython==2.1.11 in /usr/local/lib/python3.6/dist-packages (from -r microdraw/requirements.txt (line[...]<jupyter_text>2 Import librariesNow that `microdraw.py` and its dependencies are installed, we can go back to Python and import them.<jupyter_code>import sys
import numpy as np
import matplotlib.pyplot as plt
sys.path.append("./microdraw")
from microdraw import microdraw as mic<jupyter_output><empty_output><jupyter_text>3 Set project detailsAlthough you can visualise and annotate individual datasets in Microdraw, the best way for collaborating is to create a Project. A project can have multiple datasets, and you can invite collaborators to work with you. Once you log in to Microdraw (using a GitHub account), you can see all the projects in which you are a collaborator in your User page.For this tutorial, we have created a project called "[squirrelmonkey](https://microdraw.pasteur.fr/project/squirrelmonkey)", which contains a single dataset: Nissl stained slices from a Squirrel monkey, available from the Brain Museum (a project from the University of Wisconsin-Madison).This project is open, but you can also create private projects. Private tokens require a token, which in our case will be just empty.<jupyter_code># The project
project="squirrelmonkey"

# The dataset within the project
source="https://microdraw.pasteur.fr/squirrelmonkey_cor_cell/squirrelmonkey_cor_cell.json"

# The access token (empty for an open project)
token=""<jupyter_output><empty_output><jupyter_text>4 Downloading and displaying annotationsWe will first download and plot a single slice, to make sure everything is working.<jupyter_code># we are downloading the first slice, slice = 0
sl = mic.download_all_regions_from_dataset_slice(source, project, 0, token)
# each slice can have multiple regions, each with a different label 
# (in our case, labels will be "Region 1", "Region 2", etc.)
regions = mic.get_regions_from_dataset_slice(sl)
for name, region in regions:
  color = mic.color_from_string(name)
  plt.fill(region[:, 0], -region[:, 1], c=color, alpha=0.5)
# Now that we know that everything is working fine, we will download
# all the annotations made by your collaborators and you (should take less than a minute)
squirrelmonkey_slices = mic.download_all_regions_from_dataset(source, project, token)
# microdraw.py has a handy function for plotting all the slices in a dataset
# which you can use to see if there is any slice that was missing, or if
# the regions are all properly labelled. If that is not the case, you can get
# back to the Microdraw website, fix the error, come back here and rerun the
# download.
mic.draw_all_dataset(squirrelmonkey_slices)<jupyter_output><empty_output><jupyter_text>5 Compute pial surface area and brain volumeNow that we have all the data, we will use it to compute surface area and brain volume. For that, we need to find out what is the resolution of our images, and the space between slices.This is one brain slice. Its size is 720x541 pixels. Using GIMP or some other tool, you can measure the size of the scale bar in the image showing the total antero-posterior length of tha squirrel monkey brain, and the scale bar in a brain slice.You also need to know that width of the annotation layer in MicroDraw is always 1000 units.With those numbers, we can compute the scale factors we need:<jupyter_code>one_cm_in_px_slice = 280 # px/cm
one_svg_unit_in_px = 720/1000 # px/u
scaling_factor_in_cm = one_svg_unit_in_px/one_cm_in_px_slice

AP_length_in_px = 205 # px
one_cm_in_px_brain = 46 # px/cm
AP_length_in_cm = 205/406 # cm
total_number_of_slices = 65 # slices
inter_slice_space_in_cm = AP_length_in_px/one_cm_in_px_brain/total_number_of_slices

print("scaling_factor: %g cm"%(scaling_factor_in_cm))
print("inter_slice_space: %g cm"%(inter_slice_space_in_cm))
from shapely.geometry.polygon import Polygon
# to check that everything is working, we will first compute
# the surface area and perimeter of the first region in the first
# slice of our dataset
name, region = regions[0]
poly = Polygon(region)
region_volume = poly.area * scaling_factor_in_cm**2 * inter_slice_space_in_cm
region_surface_area = poly.length * scaling_factor_in_cm

print("region_volume:", region_volume)
print("region_surface_area:", region_surface_area)
# now we do that for all the dataset
total_volume = 0
total_surface_area = 0
for sl in squirrelmonkey_slices:
  for _, region in regions:
    poly = Polygon(region)
    region_volume = poly.area * scaling_factor_in_cm**2 * inter_slice_space_in_cm
    region_surface_area = poly.length * scaling_factor_in_cm
    total_volume += region_volume
    total_surface_area += region_surface_area

print("total_volume:", total_volume)
print("total_surface_area:", total_surface_area)<jupyter_output>total_volume: 0.046779072340436594
total_surface_area: 9.035293134687818<jupyter_text>6 ConclusionThat's it for today! Now you know how to use MicroDraw and microdraw.py to work collaboratively on histological data. As you see, there is a lot that you can do only from the Web browser.Let's stay in touch! We are happy to show you our projects, to help you set up your own projects. We'll be delighted if you would like to help us improve MicroDraw either by coding or by testing it.Thank you all! 😊<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07166442460560107, 'fraction_numerical': 0.02164608046960988, 'mean_word_length': 5.374123148869836, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '210452', 'license': 'MIT', 'n_tokens_mistral': 2521, 'n_tokens_neox': 2355, 'n_words': 987, 'path': 'MicroDraw_SquirrelMonkey.ipynb', 'repo_name': 'neuroanatomy/microdraw-tutorials', 'size': '8201'}