<jupyter_start><jupyter_text>Map area of objects in tilesIn this notebook, we will segment nuclei in tiles and measure their area. We will then save the resulting are area map again as tiles in a zarr file. This strategy can be used to process data that as a whole does not fit in computer memory.<jupyter_code>import zarr
import dask.array as da
import numpy as np
from skimage.io import imread
import pyclesperanto_prototype as cle
from pyclesperanto_prototype import imshow
from numcodecs import Blosc<jupyter_output><empty_output><jupyter_text>For demonstration purposes, we use a dataset that is provided by <NAME>, OncoRay, University Hospital <NAME>, TU Dresden. The dataset is licensed [License: CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/). We are using a cropped version here that was resaved a 8-bit image to be able to provide it with the notebook. You find the full size 16-bit image in CZI file format [online](https://zenodo.org/record/4276076.YX1F-55BxaQ).<jupyter_code>image = imread('../../data/P1_H_C3H_M004_17-cropped.tif')[1]

# for testing purposes, we crop the image even more.
# comment out the following line to run on the whole 5000x2000 pixels
image = image[1000:1500, 1000:1500]

#compress AND change the numpy array into a zarr array
compressor = Blosc(cname='zstd', clevel=3, shuffle=Blosc.BITSHUFFLE)

# Convert image into zarr array
chunk_size = (100, 100)
zarray = zarr.array(image, chunks=chunk_size, compressor=compressor)

# save zarr to disk
zarr_filename = '../../data/P1_H_C3H_M004_17-cropped.zarr'
zarr.convenience.save(zarr_filename, zarray)<jupyter_output><empty_output><jupyter_text>Object area maps in tilesDask brings built-in support for the zarr file format. We can create dask arrays directly from a zarr file.<jupyter_code>zarr_image = da.from_zarr(zarr_filename)
zarr_image<jupyter_output><empty_output><jupyter_text>We can apply image processing to this tiled dataset directly.<jupyter_code>def area_map(image):
    """
    Label objects in a binary image and produce a pixel-count-map image.
    """
    print("Processing image of size", image.shape)
    
    labels = cle.voronoi_otsu_labeling(image, spot_sigma=3.5)
    result = cle.pixel_count_map(labels)
    
    print(result.shape)
    
    return np.asarray(result)<jupyter_output><empty_output><jupyter_text>Testing tiled image processingWe should test our area mapping algorithm on a single tile. Actually, in a real scenario, the image processing workflow is developed on individual tiles, e.g. in a notebook like this one. As soon as we are sure that the algorithm works, we can apply it to all tiles.<jupyter_code>test_image = image[100:200,100:200]

imshow(test_image)

test_result = area_map(test_image)

imshow(test_result, colorbar=True)<jupyter_output><empty_output><jupyter_text>Applying the tiled image processing to a zarr-backed datasetApplying the function to our zarr dataset will also result in a dask array.<jupyter_code>overlap_width = 30

tile_map = da.map_overlap(area_map, zarr_image, depth=overlap_width, boundary=0)

tile_map<jupyter_output>Processing image of size (0, 0)
Processing image of size (1, 1)
(1, 1)
Processing image of size (0, 0)<jupyter_text>Before we can start the computation, we need to deactivate asynchronous execution of operations in pyclesperanto. [See also related issue](https://github.com/clEsperanto/pyclesperanto_prototype/issues/163).<jupyter_code>cle.set_wait_for_kernel_finish(True)<jupyter_output><empty_output><jupyter_text>When we invoke saving the results to disk, the processing will happen on individual tiles.<jupyter_code>processed_zarr_filename = '../../data/P1_H_C3H_M004_17-processed.zarr'

tile_map.to_zarr(processed_zarr_filename, overwrite=True)<jupyter_output>Processing image of size (160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
Processing image of size Processing image of size (160, 160)
(160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
(160, 160)
Processing image of size (160, 160)
(160, 160)
(160, 160)
(160, 160)
Processing image of size Processing image of size (160, 160)
(160, 160)
(160, 160)
(160, 160)
(160, 160)
Processing image of size (160, 160)
Processing image of size(160, 160) 
(160, 160)
Processing image of size (160, 160)
(160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
Processing image of size (160, 160)
(160, 160)
Processing image of size (160, 160)
(160, 160)
Processing image of size (160, 160)
(160, 160)
(160, 160)
(160, 160)
(160, 160)Processing image of sizeProcessing image of size(160, 160)
 (160, 160)
 
(160, 1[...]<jupyter_text>Loading zarrJust for demonstration purposes, we will load the zarr backed tiled image and visualize it. When working with big data, this step might not be possible.<jupyter_code>zarr_result = da.from_zarr(processed_zarr_filename)
zarr_result
result = zarr_result.compute()

cle.imshow(result)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08857030636292224, 'fraction_numerical': 0.06480754124116261, 'mean_word_length': 5.580103359173127, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '387917', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1900, 'n_tokens_neox': 1623, 'n_words': 588, 'path': 'docs/32_tiled_image_processing/tiled_area_mapping.ipynb', 'repo_name': 'rayanirban/BioImageAnalysisNotebooks', 'size': '5112'}