<jupyter_start><jupyter_text>Combine individual z-layers and channels to multichannel z-stacks (Setup)This is a setup notebook for adjusting parameters for [run_images_to_stack.ipynb](run_images_to_stack.ipynb).Run this notebook cell-by-cell and follow the instructions. Requirements- A folder with images to convertIndividual channels and z-layers must be clearly tagged in the filename, and the naming pattern must be consistent across the dataset. The naming pattern should follow the following scheme: `[SAMPLENAME][Z_CODE][Z][CHANNEL_CODE][CHANNEL].tif` or `[SAMPLENAME][CHANNEL_CODE][CHANNEL][Z_CODE][Z].tif`**Example**: `"condition1/my_sample_position-3_Z00_C01.tif”``[SAMPLENAME]` may include subfolders but must be the same for all files that will be combined. Config The following code imports and declares functions used for the processing:<jupyter_code>#################################
#  Don't modify the code below  #
#################################

import os
import numpy as np
import intake_io
from am_utils.utils import walk_dir
from punctatools.lib.convert import load_random_dataset, check_metadata, get_number_of_stacks
from punctatools.lib.utils import save_parameters<jupyter_output>2022-04-06 16:05:49,811 [INFO] WRITING LOG OUTPUT TO /home/amedyukh/.cellpose/run.log<jupyter_text>Set up the data and parameters Please provide data paths:`input_dir`: folder with images to be converted`output_dir`: folder to save results<jupyter_code>input_dir = "../example_data/slices"
output_dir = "../test_output/stacks"<jupyter_output><empty_output><jupyter_text>The following code lists all image files in the input directory:<jupyter_code>#################################
#  Don't modify the code below  #
#################################

input_dir = os.path.realpath(input_dir)

samples = walk_dir(input_dir)

print(f'{len(samples)} images were found:')
print(np.array(samples))<jupyter_output>30 images were found:
['/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z20_C0.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z20_C1.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z20_C2.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z21_C0.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z21_C1.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z21_C2.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z22_C0.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/sample1_position1_Z22_C1.tif'
 '/research/sharedresources/cbi/common/Anna/codes/punctatools/example_data/slices/[...]<jupyter_text>Please specify codes for channel and z-position:Specify the sequence of characters that precedes the channel and z-position numbering, including the separator (e.g. "_") that precedes the channel code.For example, if the image name is "my_experiment_position1_C0_z001.tif", the codes should be as follows:`channel_code` = "_C"`z_position_code` = "_z"<jupyter_code>channel_code = "_C"
z_position_code = "_Z"<jupyter_output><empty_output><jupyter_text>The following code returns the number of stacks that were foundIf executing this cell results in an error, make sure the channel and z-position codes are provided correctly. Note the the codes are case-sensitive<jupyter_code>#################################
#  Don't modify the code below  #
#################################

n = get_number_of_stacks(input_dir, channel_code=channel_code, z_position_code=z_position_code)
n<jupyter_output><empty_output><jupyter_text>The following code checks image metadata<jupyter_code>#################################
#  Don't modify the code below  #
#################################

dataset, spacing = check_metadata(input_dir, channel_code=channel_code, z_position_code=z_position_code)
print('The following voxel size was detected:')
if 'z' in dataset.dims:
    dims = ['z', 'y', 'x']
else:
    dims = ['y', 'x']
for s, c in zip(spacing, dims):
    print(rf"{c}: {s}")<jupyter_output>The following voxel size was detected:
z: None
y: 0.11000000110000001
x: 0.11000000110000001<jupyter_text>Please specify correct voxel size Specify the correct voxel size (pixel size and z-stepping) in microns if the above values are wrong.Keep `None`, if the value loaded from the metadata is correct.<jupyter_code>z = 0.2
y = None
x = None<jupyter_output><empty_output><jupyter_text>The following code loads a random stack with correct voxel size<jupyter_code>#################################
#  Don't modify the code below  #
#################################

if 'z' in dataset.dims:
    spacing = [z, y, x]
else:
    spacing = [y, x]
dataset = load_random_dataset(input_dir, channel_code=channel_code, 
                              z_position_code=z_position_code, spacing=spacing)
dataset<jupyter_output><empty_output><jupyter_text>Run the full processingThe parameters set up in this notebook will be used to convert images in batch. Please provide the file name to save the parametersIf only a filename is provided, the parameter file will be saved in the same directory with this notebook. Provide the full path to save the parameters to a different directory.Default is `parameters.json`<jupyter_code>parameter_file = 'parameters.json'<jupyter_output><empty_output><jupyter_text>Please specify the number of processes to run in parallel`n_jobs`: number of processes to run in parallel. Set according to your workstation resources. Decrease if the processing runs out of memory. (Default is 8)<jupyter_code>n_jobs=8<jupyter_output><empty_output><jupyter_text>The following code saves the parametersTo run the full processing, run the notebook [run_images_to_stack.ipynb](run_images_to_stack.ipynb) with the parameter file specified here. Alternatively run the [run_images_to_stack.py](../scripts/run_images_to_stack.py) script:``python run_images_to_stack.py -p ``<jupyter_code>#################################
#  Don't modify the code below  #
#################################

spacing = list(intake_io.get_spacing(dataset))

params = dict(
    parameter_file_path=os.path.realpath(parameter_file),
    raw_dir=os.path.realpath(input_dir),
    converted_data_dir=os.path.realpath(output_dir),
    spacing=spacing,
    channel_code=channel_code,
    z_position_code=z_position_code,
    n_jobs=n_jobs
)
params = save_parameters(params, parameter_file)
    
params<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14375, 'fraction_numerical': 0.01636904761904762, 'mean_word_length': 6.725287356321839, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 42, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '661898', 'license': 'Apache-2.0', 'n_tokens_mistral': 2190, 'n_tokens_neox': 2050, 'n_words': 530, 'path': 'notebooks/setup_images_to_stack.ipynb', 'repo_name': 'stjude/punctatools', 'size': '6720'}