<jupyter_start><jupyter_text>Importing librariesNotes for user:1. No change is needed in this cell.2. Wait for `Initialisation is done..` is printed before continuing.3. To run this cell section, click `>| Run` button.4. If you want to clear the output to run pristine code, choose `Kernel > Restart & Clear Output`.5. If you want to run all the active cells, choose `Kernel > Restart & Run All`.<jupyter_code>from LOTS_IM_GPU_FUNction import *

print('Check OpenCV version: ' + cv2.__version__ + '\n')
print(cuda.current_context().get_memory_info())
print('Initialisation is done..\n')<jupyter_output>Check OpenCV version: 3.4.1

_MemoryInfo(free=7498760192, total=8513978368)
Initialisation is done..<jupyter_text>List of MRI data that will be processed using IAM* Below is a list of variables that needs to be changed manually.* If you need more explanation of these parameters, please see [this documentation in GitHub](https://github.com/febrianrachmadi/lots-iam-gpu23-changing-softwares-parameters).* Note: To run this cell section, click `>| Run` button.<jupyter_code>## General output full path (note to user: you can change this variable)
## This variable will be overriden if input files are .mat
output_filedir = "/mnt/storage/MRI_dataset/LOTS_IM_results"


output_filedir = "/mnt/Storage/ADNI_20x3_2015/LOTS_IM_results_multi"

## Set setting of LOTS-IM's calculation
## Default: True --> 3D LOTS-IM (volume based calculation)
## Otherwise (False): 2D LOTS-IM (slice based calculation)
set_3d = True

## Size of source and target patches.
## Must be in the form of python's list data structure.
## Default: patch_size = [1,2,4,8]
patch_size = [1,2,4,8]

## Weights for age map blending produced by different size of source/target patches
## Must be in the form of python's list data structure.
## Its length must be the same as 'patch_size' variable.
## Default: blending_weights = [0.65,0.2,0.1,0.05]
blending_weights = [0.65,0.2,0.1,0.05]

## Used only for automatic calculation for all number of samples
## NOTE: Smaller number of samples makes computation faster (please refer to the manuscript).
## Samples used for IAM calculation 
## Default: num_samples_all = [512]
num_samples_all = [64,128]
## Uncomment line below and comment line above if you want to run all different number of samples 
# num_samples_all = [64, 128, 256, 512, 1024, 2048]

## Weight of distance function to blend maximum difference and average difference between source
## and target patches. Default: alpha=0.5. Input value should be between 0 and 1 (i.e. floating).
alpha = 0.5

## Thresholds the target patches to prevent including patches containing hyper-intensities.
## Default : threshold_patches = None.
# thrsh_patches = 0.05
thrsh_patches = 0.05

## Save JPEG outputs
## Default: save_jpeg = True
save_jpeg = True

## Delete all intermediary files/folders, saving some spaces in the hard disk drive.
## Default: delete_intermediary = True
delete_intermediary = True

print("--- PARAMETERS - CHECKED ---")
print('Patch size(s): ' + str(patch_size))
print('Number of samples (all): ' + str(num_samples_all))
print('Save JPEGs? ' + str(save_jpeg))
print("--- PARAMETERS - CHECKED ---")

print('--\nReady..')<jupyter_output>--- PARAMETERS - CHECKED ---
Patch size(s): [1, 2, 4, 8]
Number of samples (all): [64, 128]
Save JPEGs? True
--- PARAMETERS - CHECKED ---
--
Ready..<jupyter_text>Run LOTS-IAM-3D-GPU by opening Nifti files first* To run this cell section, click `>| Run` button.* If you need help, please see run: `help(iam_lots_gpu_mat_compute)`<jupyter_code>'''
Name of csv file (note to user: you can change this variable)

Default format of the CSV input file (NOTE: spaces are used to make the format clearer):

    input_file_path, output_folder_path, patient_code

Example (NOTE: spaces are used to make the format clearer):

    W:/DMP01/V1/flair.mat,W:/results_lots_iam_3d,DMP01_V1
    W:/DMP01/V2/flair.mat,W:/results_lots_iam_3d,DMP01_V2
    W:/DMP01/V3/flair.mat,W:/results_lots_iam_3d,DMP01_V3

'''
csv_filename = "ADNI_small_test_pipeline.csv"

with open(csv_filename, newline='') as csv_file:
    num_subjects = len(csv_file.readlines())
    print('Number of subject(s): ' + str(num_subjects))

''' For each experiments using different number of target patches.. '''
for ii_s in range(0, len(num_samples_all)):
    with open(csv_filename, newline='') as csv_file:
        reader = csv.reader(csv_file)
    
        ''' Set output directory based on the number of target patches '''
        num_samples = num_samples_all[ii_s]
        set_output_dir = output_filedir + '_' + str(num_samples) + 's'
    
        timer_idx = 0
        elapsed_times_all = np.zeros((num_subjects))
        elapsed_times_patch_all = []
        for row in reader:
            data = row[1]
            print('--\nNow processing data: ' + data)
            
            '''
            Read MRI files and do pre-processing.
            
            NOTE: The current version of demo reads the required MRI files and
            performs pre-processing outside the main LOTS-IM function.
            '''

            inputSubjectDir = row[0] + '/' + row[1]
            print('Input filename (full path): ' + inputSubjectDir)

            mri_nii = nib.load(row[2])
            icv_nii = nib.load(row[3])
            csf_nii = nib.load(row[4])

            mri_data = np.squeeze(mri_nii.get_data())
            icv_data = np.squeeze(icv_nii.get_data())
            csf_data = np.squeeze(csf_nii.get_data())
            print(' -> Loading FLAIR + ICV + CSF: OK!')

            ''' Make sure that all brain masks are binary masks, not probability '''
            bin_tresh = 0.5
            icv_data[icv_data > bin_tresh] = 1
            icv_data[icv_data <= bin_tresh] = 0
            csf_data[csf_data > bin_tresh] = 1
            csf_data[csf_data <= bin_tresh] = 0

            ''' Read and open NAWM data if available '''
            nawm_available = len(row) > 5 and row[5]
            if nawm_available:
                print(" -> NAWM mask is avalilable!")
                print(" --> " + row[5])
                nawm_nii = nib.load(row[5])
                nawm_data = np.squeeze(nawm_nii.get_data())
                nawm_data[nawm_data > bin_tresh] = 1
                nawm_data[nawm_data <= bin_tresh] = 0
            else:
                print(" -> Cortex mask is NOT avalilable!")

            ''' Read and open Cortex data if available '''
            cortex_available = len(row) > 6 and row[6]
            if cortex_available:
                print(" -> Cortex mask is avalilable!")
                print(" --> " + row[6])
                cortex_nii = nib.load(row[6])
                cortex_data = np.squeeze(cortex_nii.get_data())
                cortex_data[cortex_data > bin_tresh] = 1
                cortex_data[cortex_data <= bin_tresh] = 0
            else:
                print(" -> Cortex mask is NOT avalilable!")

            del csf_nii   # Free memory

            ''' ICV Erosion '''
            print(' -> ICV Erosion -- note: skimage library')
            print(' -> ICV shape:' + str(icv_data.shape) + '\n')
            for ii in range(0, icv_data.shape[2]):
                kernel = kernel_sphere(7)
                icv_data[:,:,ii] = skimorph.erosion(icv_data[:,:,ii],kernel)
                kernel = kernel_sphere(11)
                icv_data[:,:,ii] = skimorph.erosion(icv_data[:,:,ii],kernel)
                if nawm_available:
                    kernel = kernel_sphere(3)
                    nawm_data[:,:,ii] = scimorph.binary_fill_holes(nawm_data[:,:,ii])
                    nawm_data[:,:,ii] = skimorph.erosion(nawm_data[:,:,ii],kernel)

            csf_data = csf_data.astype(bool)
            csf_data = ~csf_data

            mask_data = np.multiply(csf_data, icv_data)
            brain_data = np.multiply(mask_data, mri_data)

            nawm_preprocessing = False
            if nawm_available and nawm_preprocessing:
                print("NAWM pre-processing..")
                nawm_data = nawm_data.astype(bool)
                mask_data = np.multiply(mask_data, nawm_data)
                nawm_data = np.int16(nawm_data)
                brain_data  = np.multiply(brain_data, nawm_data)

            if cortex_available:
                cortex_data = cortex_data.astype(bool)
                cortex_data = ~cortex_data
                mask_data = np.multiply(mask_data, cortex_data)
                cortex_data = np.int16(cortex_data)
                brain_data  = np.multiply(brain_data, cortex_data)
            ''' Pre-processing is done '''
            
            ''' Call LOTS-IM GPU for each MRI data '''
            timer_data, timer_patches = lots_im_function_compute(
                                    mri_code         = data,
                                    input_mri_data   = brain_data,
                                    output_filedir   = set_output_dir,
                                    nifti_header     = mri_nii.affine,
                                    set_3d           = set_3d,
                                    patch_size       = patch_size,
                                    blending_weights = blending_weights,
                                    num_sample       = num_samples,
                                    alpha            = alpha,
                                    thrsh_patches    = thrsh_patches,
                                    save_jpeg        = save_jpeg,
                                    delete_intermediary = delete_intermediary)

            ''' Save the elapsed time for each MRI data '''
            elapsed_times_all[timer_idx] = timer_data
            elapsed_times_patch_all.append(timer_patches)
            timer_idx += 1

        ''' Save the elapsed time for all MRI data '''
        ''' Save all elapsed times '''
        print("Save elapsed time at: " + set_output_dir)
        sio.savemat(set_output_dir + '/elapsed_times_all_test.mat',
                    {'elapsed_times_all':elapsed_times_all})
        sio.savemat(set_output_dir + '/elapsed_times_patch_all_test.mat',
                    {'elapsed_times_patch_all':elapsed_times_patch_all})

help(lots_im_function_compute)<jupyter_output>Help on function lots_im_function_compute in module LOTS_IM_GPU_FUNction:

lots_im_function_compute(mri_code='', input_mri_data='', output_filedir='', nifti_header='', set_3d=True, patch_size=[1, 2, 4, 8], blending_weights=[0.65, 0.2, 0.1, 0.05], num_sample=512, alpha=0.5, thrsh_patches=None, save_jpeg=True, delete_intermediary=True)
    FUNCTION'S SUMMARY:
    Main function of the LOTS-IM algorithm running on GPU. This function produces (i.e. saving)
    irregularity maps that indicate level of irregularity of voxels in brain FLAIR MRI, either
    in slice (2D) or volume (3D). This function read MRI data from a single subject (i.e., only
    1 MRI data for each run). Please note that the data is retrieved as a 3D numpy array (i.e.,
    whole MRI volume).
    
    By default, the irregularity maps are calculated by using four different sizes of source/target
    patches (i.e. 1x1, 2x2, 4x4, and 8x8) and 64 target samples. Furthermore, all intermediary
    files are saved in .mat (Matla[...]
{'fraction_non_alphanumeric': 0.1020226320948053, 'fraction_numerical': 0.018889779916243427, 'mean_word_length': 2.391961317618616, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 31, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '122265', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3713, 'n_tokens_neox': 3457, 'n_words': 956, 'path': 'LOTS_IM_GPU_FUNction_release.ipynb', 'repo_name': 'iboele/lots-iam-gpu', 'size': '11223'}