<jupyter_start><jupyter_text>Selection of number of filters and kernel size in convolutional layers To study the impact of the kernel size and the initial number of filters in the UNet model on the segmentation quality and to choose the optimal value. * [The parameters studied](The-parameters-studied)* [Parameters Value Area](Parameters-Value-Area)* [Metrics](Metrics)* [Dataset loading](Dataset-loading)* [Creating a model template](Creating-a-model-template)* [Creating a research object](Creating-a-research-object)* [Results](Results)* [Conclusion](Conclusion)  The parameters studiedKernel size and number of filters in the first convolutional layer (the subsequent layers have twice the number of filters in relation to the previous layer). Parameters Value Area* __A number of filters in the first convolutional layer::__ 4, 12, 32* __Kernel size:__ 3х3, 5x5 MetricsThe average value of the IOU (intersection over union) on the test set.<jupyter_code>import os
import sys
sys.path.insert(0, os.path.join("..", "..", ".."))

import numpy as np

from batchflow import Pipeline, B, V, C, F, L, NumpySampler as NS
from batchflow.opensets import MNIST
from batchflow.models.tf import UNet
from batchflow.research import Option, Research, Results

from src import MnistBatch, show_research, show_samples

%env CUDA_VISIBLE_DEVICES='6,7'<jupyter_output>env: CUDA_VISIBLE_DEVICES='6,7'<jupyter_text>Dataset loading For training and model performance assessment we generate images of shape 60x60 on which a MNIST digit is randomly located. Also we add random noise cropped from other MNIST images.A label data for every image is a binary array of the same shape as an image with the value "1" if the corresponding pixel on the image represents mnist image rectangle and value "0" otherwise.Data is generated on each iteration of training and test pipelines.Load original MNIST dataset:<jupyter_code>mnist_ds = MNIST(batch_class=MnistBatch)<jupyter_output><empty_output><jupyter_text>A position of mnist image's bottom left corner is randomly sampled from [2, 40].Create uniform sampler to generate random digit positions:<jupyter_code>SHAPE = 60
ns = NS('uniform', dim=2, low=2, high=(SHAPE - 20)).apply(lambda s: s.astype(np.int))<jupyter_output><empty_output><jupyter_text>Define preprocessing pipeline which consists of:* Creating a 60x60 image with a MNIST image located in a random position.* Adding noise cropped from other images in the batch.<jupyter_code>prep_pipeline = (Pipeline()
                 .create_big_img(coord=L(ns.sample)(size=F(len)), 
                                 shape=SHAPE, 
                                 src=['images', 'labels'], 
                                 dst=['big_img', 'masks'])
                 .add_noize(num_parts=50)
                 .to_array(src='masks', dst='masks')
                 .to_array(src='big_img', dst='big_img')
)<jupyter_output><empty_output><jupyter_text>Examples of dataset items (top row) and corresponding labels (bottom row):<jupyter_code>batch = (prep_pipeline << mnist_ds.train).next_batch(5)
show_samples([batch.big_img[:, :, :, 0], batch.masks[:, :, :, 0]], row_titles=['Images', "Masks"])<jupyter_output><empty_output><jupyter_text>Creating a model templateWe use UNet model with number of blocks NUM_BLOCKS = 3 and different number of filters in the first convolutional layer.<jupyter_code>NUM_BLOCKS = 3
model_config = {
        'inputs/images/shape': tuple([SHAPE, SHAPE, 1]),
        'inputs/masks/shape': tuple([SHAPE, SHAPE, 1]),
        'initial_block/inputs': 'images',
    
        'initial_block/filters': C('initial_block_filters'), 
        'initial_block/kernel_size': C('kernel_size'),
        'body/encoder/num_stages': NUM_BLOCKS - 1,
        'body/encoder/blocks/filters': C('encoder_filters'), 
        'body/encoder/blocks/kernel_size': C('kernel_size'),
        'body/decoder/blocks/filters': C('decoder_filters'),
        'body/decoder/blocks/kernel_size': C('kernel_size'),
        'head/filters': C('initial_block_filters'),
        'head/kernel_size': C('kernel_size'),
    
        'loss': 'bce',
        'optimizer': 'Adam',
        'output': ['sigmoid']
}<jupyter_output><empty_output><jupyter_text>Define train and test pipeline and set parameters of lazy run with BATCH_SIZE = 40:<jupyter_code>BATCH_SIZE = 40
train_pipeline = ((prep_pipeline << mnist_ds.train)
                  .init_variable('loss', init_on_each_run=list)
                  .init_model('dynamic', UNet, 'unet',
                              config=model_config)
                  .train_model('unet', 
                               fetches='loss', 
                               images=B('big_img'), masks=B('masks'),
                               save_to=V('loss'))
                  .run_later(BATCH_SIZE, n_epochs=None)
)

test_pipeline = ((prep_pipeline << mnist_ds.test)
                 .init_variable('predictions', init_on_each_run=list)
                 .init_variable('metrics', init_on_each_run=None)
                 .import_model('unet', C('import_from'))
                 .predict_model('unet', fetches='sigmoid', 
                                images=B('big_img'),
                                save_to=V('predictions'))
                 .gather_metrics('segmentation',
                                 targets=B('masks'),
                                 predictions=V('predictions'),
                                 fmt='proba', save_to=V('metrics', mode='w'))
                 .run_later(BATCH_SIZE, n_epochs=None)
)<jupyter_output><empty_output><jupyter_text>Creating a research objectSet the parameters for research execution:For each parameter set perform NUM_ITERS=450 iterations of train pipeline with model performance evaluation every single iteration. Repeat this procedure NUM_REPEAT=4 times.Results of the research execution will be saved in the directory __two_hyper_research__<jupyter_code>ibf = [4, 12, 32]

opts = (Option.product(Option('initial_block_filters', ibf), 
                      Option('encoder_filters', [[f0 * 2* 2**i for i in range(NUM_BLOCKS - 1)] for f0 in ibf]),
                      Option('decoder_filters', [[f0 * 2**i for i in range(NUM_BLOCKS - 2, -1, -1)] for f0 in ibf]))
       * Option('kernel_size', [3, 5]))
list(opts.gen_configs())
research = (Research()
            .add_pipeline(train_pipeline, variables='loss', name='train')
            .add_pipeline(test_pipeline, name='test', execute=10, run=False,
                          import_from='train', variables='metrics')
            .get_metrics(pipeline='test', metrics_var='metrics', metrics_name='iou',
                         returns='iou', execute=10)
            .add_grid(opts))<jupyter_output><empty_output><jupyter_text>Run research:<jupyter_code>N_REPS = 4
N_ITERS = 450
research_name = 'two_hyper_research'
research.run(n_reps=N_REPS, n_iters=N_ITERS, name=research_name, bar=True,  workers=2, gpu=[6,7])<jupyter_output>Research two_hyper_research is starting...<jupyter_text>Results<jupyter_code>df = Results(path=research_name).load(use_alias=True)<jupyter_output><empty_output><jupyter_text>Train loss and test IOU vs a number of iterations for each parameters set:<jupyter_code>show_research(df, layout=['train/loss', 'test_metrics/iou'], average_repetitions=True, rolling_window=25)
res = research.load_results(names='test_metrics', iterations=449)
res.groupby(['initial_block_filters', 'kernel_size'])['iou'].mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11444652908067542, 'fraction_numerical': 0.01139104797641383, 'mean_word_length': 3.542300669506999, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '605871', 'license': 'Apache-2.0', 'n_tokens_mistral': 2310, 'n_tokens_neox': 2185, 'n_words': 573, 'path': 'model_description/additional_examples/parameters_estimation/num_filters_and_kernel_size.ipynb', 'repo_name': 'gazprom-neft/standards', 'size': '7462'}