<jupyter_start><jupyter_text>CSBDeep & Noise2Noise an Astrophotography ExampleWe use the [CSBDeep](http://csbdeep.bioimagecomputing.com/) training framework which was published with [CARE](https://www.nature.com/articles/s41592-018-0216-7). Furthermore we use the [Noise2Noise](https://arxiv.org/abs/1803.04189) training principle first described by Lehtinen _et al._Noise2Noise states that multiple noisy observation are sufficient to train content aware image restoration networks. In other words, no ground truth data is required. This opens the door to many new applications where the acquisition of ground truth data is impossible or very time consuming. In this example we will train a content aware image restoration network with multiple subs from an astrophotography session.The possibility of using this learning model for astrophotography is discussed by the authors in this video: https://www.youtube.com/watch?v=dcV0OfxjrPQThe example here is based on learnings taken from examples in the CSBDeep repo `examples` and from the basic Noise2Noise example provided by <NAME> in the repo [Noise2Noise-with-CSBDeep](https://github.com/juglab/Noise2Noise-with-CSBDeep). Additionally the CSBDeep team have provided an great video about their framework here: https://www.youtube.com/watch?v=ipp0mxfjhwYOther contributions have been from learnings acquired from various astro forum posts on the topic of deep learning:* https://www.cloudynights.com/topic/789780-deep-learning-denoising-for-astrophotography/* https://www.cloudynights.com/topic/657928-deep-learning-for-random-noise-attenuation/*Enhancements have been provided by <NAME> as code changes to the base CDBDeep repo.*<jupyter_code># Allow reloading of CSBDeep modules following any code changes
%reload_ext autoreload
%autoreload 2

# A couple required imports
import numpy as np
import matplotlib.pyplot as plt
from tifffile import imread
from pathlib import Path<jupyter_output><empty_output><jupyter_text>Pre-processing Training Data and SaveWe have a stack of sub frames in the `data/astro/train` folder, acquired with the same optical train and equipment.All images have been aligned and calibrated and saved in the tiff format.The example data can be downloaded [here](https://1drv.ms/u/s!AvWEkn9Anb_Nq9Aw52Xs3LuYEcq_rg?e=EexXxL)Place the train images in the `data/astro/train` folder.<jupyter_code># Setup Data Parameters:
# Root Data path
basepath=Path('data/astro')
# Train Data path/s
source_dirs=['train']
# Image file pattern. Note: only formats supported by imread currently
pattern='*.tiff'
# Image patch size
patchsize=64
# Training data output savefile path & name
training_data_name='TrainData_p64_nPERC'
training_data_filename=training_data_name + '.npz'
save_file=basepath/training_data_filename
# Make an estimate of the number of non-overlapping patches for the images we have, sampling from the first we find.
sampleimage = imread(list((basepath/source_dirs[0]).glob(pattern))[0])
n_patches_per_image=np.int(sampleimage.shape[0]/patchsize)*np.int(sampleimage.shape[1]/patchsize)
from csbdeep.data import RawData, create_patches, norm_percentiles, norm_reinhard
from csbdeep.data import NoPreProcessor, ReinhardPreProcessor

# Load image pairs for Noise2Noise processing, each image paired against every other at most once.
raw_data = RawData.from_folder_n2n(basepath, source_dirs=source_dirs, axes='YXC', pattern=pattern, preprocessor=NoPreProcessor())

# Create patch data from image pairs with parameters,
# normalization set as norm_percentiles() by default, optionally set to None, norm_reinhard() or other custom
X, Y, XY_axes = create_patches(
    raw_data, 
    patch_size=(patchsize,patchsize,3),
    normalization=norm_percentiles(),
    n_patches_per_image=n_patches_per_image,
    save_file=save_file,
    patch_filter=None,
    overlap=False)

print('Source Data Shape =', X.shape)
print('Target Data Shape =', Y.shape)
print('Data Axes =', XY_axes)
from csbdeep.utils import plot_some

# We can visualize the resulting patch input and target pairs.
for i in range(2):
    plt.figure(figsize=(32,4))
    sl = slice(16*i, 16*(i+1)), 0
    plot_some(X[sl],Y[sl],title_list=[np.arange(sl[0].start,sl[0].stop)])
    plt.show()

# Clear loaded data from memory, as an alternative axes order is required for subsequent steps.
del X,Y,XY_axes<jupyter_output><empty_output><jupyter_text>Load Training DataHere we load the training data from a save file created in earlier steps.As data is loaded we can also split the data into training `X,Y` and validation `X_val,Y_val` sets.<jupyter_code>from csbdeep.io import load_training_data
# Train/Validation split %
validation_split=0.1
# Load saved training data to memory from save file
(X,Y), (X_val,Y_val), axes = load_training_data(save_file, validation_split=validation_split, verbose=True)

print('Source Train Data Shape =', X.shape)
print('Target Train Data Shape =', Y.shape)
print('Source Val Data Shape =', X_val.shape)
print('Target Val Data Shape =', Y_val.shape)<jupyter_output><empty_output><jupyter_text>Configure and Train the Learning ModelHere we configure the training model parameters. Training will be done for each color channel separately and saved with individual names based on:* `training_data_name`* `model_base_name`* `channel_name[i]` Training debug toolsYou can monitor the progress during training with [TensorBoard](https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard) by starting it from the current working directory:    $ tensorboard --logdir=.Then connect to [http://localhost:6006/](http://localhost:6006/) with your browser.<jupyter_code># Importing the deep learning framework
from csbdeep.models import Config, CARE

# Since we are training each channel separately: n_channel_in, n_channel_out = 1, 1
# Probabilistic training will be used as this yields better results
# Reccomended that train_steps_per_epoch set at 200-400, and train_epochs at 100-200 for good results.
config = Config('SYXC', n_channel_in=1, n_channel_out=1, unet_kern_size=3, probabilistic=True, train_steps_per_epoch=100, train_epochs=50)

# Give a name for the model
model_base_name = '_ASTRO_01_01'

model_name = training_data_name + model_base_name
channel_names=['R', 'G', 'B']
for i, channel in enumerate(channel_names):
    full_model_name = model_name + '_' + channel
    
    # Create the Learning Model from the CARE framework with configuration
    model = CARE(config, name=full_model_name, basedir='models')
    
    # Train the model and capture history
    history = model.train(
        X[:,:,:,i,np.newaxis], 
        Y[:,:,:,i,np.newaxis], 
        (X_val[:,:,:,i,np.newaxis], Y_val[:,:,:,i,np.newaxis]))
    
    # Save the model
    model.export_TF()<jupyter_output><empty_output><jupyter_text>Load and De-noise an Image Here we load an example image to de-noise using the trained set of RGB models.The image to be de-noised must be normalized in the same way as the training data.Then for each channel the model is used to predict the de-noised output and these are then saved as a single RGB image.The example test image used can be downloaded [here](https://1drv.ms/u/s!AvWEkn9Anb_Nq9Aw52Xs3LuYEcq_rg?e=EexXxL).Place the test image in the `data/astro/test` folder.<jupyter_code># Specify the test file
test_file_name='CrescentNebula-NoSt-Deep.tiff'

testfilepath=basepath/'test'/test_file_name
x = imread(testfilepath)
testaxes = 'YX'

print('Test Image size =', x.shape)
print('Test Image axes =', testaxes)
from csbdeep.data import PercentileNormalizer, PadAndCropResizer, ReinhardNormalizer, NoNormalizer

channel_names=['R', 'G', 'B']
output_denoised = []
for i, channel in enumerate(channel_names):
    full_model_name = model_name + '_' + channel
    
    # Load the model for the specific channel
    print("Loading model:", full_model_name)
    model = CARE(config=None, name=full_model_name, basedir='models')

    # Predict/de-noise the image channel with the corresponding trained model
    # Default PercentileNormalizer is used to match the normalization used to train the model
    output_denoised.append(
        model.predict(x[:,:,i],testaxes, normalizer=PercentileNormalizer(), resizer=PadAndCropResizer())
    )
# Load astropy library for saving the de-noised image in fits format
from astropy.io import fits

output_file_name = model_name + '_RGB_' + Path(test_file_name).stem + '.fits'
output_file_path = basepath/'test'/output_file_name
hdu = fits.PrimaryHDU(output_denoised)
hdul = fits.HDUList([hdu])
hdul.info()
hdul.writeto(output_file_path)
print("Output file saved:", output_file_path)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09058264797868644, 'fraction_numerical': 0.013205143055716437, 'mean_word_length': 5.693023255813953, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 10, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '174464', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2742, 'n_tokens_neox': 2626, 'n_words': 852, 'path': 'AstroNoise2Noise.ipynb', 'repo_name': 'tralamazza/astro-csbdeep', 'size': '8652'}