<jupyter_start><jupyter_text>3D Lables EDAExplor file structure, data structure and lables of our 3D images of mouse skulls and explore some of the issue facing the product development.The image files are in their original .mnc format which is an AutoCAD Compiled Menu file, while the keypoints files are in .tag format.---We are using the `nibabel` package to read the `.mnc` files<jupyter_code>import matplotlib.pyplot as plt
import nibabel as nib
import numpy as np
img = nib.load("/Users/michaeldac/Code/CUNY/698/Skulls/475.mnc")<jupyter_output><empty_output><jupyter_text>Lets get the type and shape of the image file.<jupyter_code>data = img.get_data()
print("The data shape is:", data.shape)
print("The type of data is:", type(data))
np.set_printoptions(precision=2, suppress=True)
print(data[0:4, 0:4, 0:4])<jupyter_output>The data shape is: (698, 397, 456)
The type of data is: <class 'numpy.ndarray'>
[[[-242.   -186.99 -304.03 -101.02]
  [ -59.98 -216.98 -267.03  -55.02]
  [  31.01   29.98 -118.01   68.97]
  [ -35.98  230.02  337.03  221.01]]

 [[-179.02  -62.    148.97  143.02]
  [ -72.02    7.98   93.98   99.02]
  [  59.02  125.    152.    146.  ]
  [  64.     -3.98  -45.98   40.99]]

 [[   8.03  128.02  128.99  -11.  ]
  [  92.01  181.01   90.02    1.02]
  [  88.99   41.98 -118.01  -69.98]
  [ 137.02   43.98 -114.99  -23.03]]

 [[-117.    -31.99  -94.99  -12.  ]
  [ 103.03   32.02 -155.98  -89.  ]
  [  -3.99   32.02 -208.   -107.98]
  [ 208.03  132.99 -178.99   26.98]]]<jupyter_text>As we can see, this particluar image has a shape of 698 x 397 x 456 voxels. Since we are dealing with three-dimensional images we will have to work with volume pixels, or voxels.-----Let's take a look at the images by plotting them. Since they are in 3d and we are using a 2d canvas, we can only look at particular slices of the 3d image.<jupyter_code>img_data = img.get_fdata()


def show_slices(slices):
    """Function to show image slices"""
    fig, axes = plt.subplot(1, len(slices), 1)
    i=0
    for s in slices:
        axes[i].imshow(slice.T, cmap="gray", origin="lower")
        i+=1

slice_0 = img_data[350, :, :]
slice_1 = img_data[:, 200, :]
slice_2 = img_data[:, :, 225]
#show_slices([slice_0, slice_1, slice_2])
# plt.suptitle("Center slices for EPI image")  # doctest: +SKIP


plt.imshow(slice_1)
plt.show()<jupyter_output><empty_output><jupyter_text>You can see that in each of the three image slices there are differences in brightness which correspond to each value in the array. The first image appears to be a top-down view of the mouse's skull.Unlike many photos these allow negative value instead of having a scale of 0-255. More invistigation needs to be done to find out what the best way to scale these for a neural network are.<jupyter_code>plt.imshow(slice_2)
plt.show()<jupyter_output><empty_output><jupyter_text>The second image looks like its a side view of the skull and the third image appears to be a view from the back of the head.<jupyter_code>plt.imshow(slice_0)
plt.show()<jupyter_output><empty_output><jupyter_text>Now let's move on to the keypoint files. We've created a `tag_parser` function to split up the original file, remove the excess, and obrain a 3d ndarray.<jupyter_code>import pandas as pd
from io import StringIO
from preprocessing import tag_parser
tags = tag_parser('/Users/michaeldac/Code/CUNY/698/Skulls/475_landmarks.tag')
tags
tags.shape
img_475 = (data, tags)
img_475_array = img_475[0]
img_476 = (data, tags)
img_475[0]
np.save('img_475.npy', img_475)
reload = np.load('img_475.npy', allow_pickle=True)
reload<jupyter_output><empty_output><jupyter_text>The 3D images are accompanied by `.tag` files that denote the `(x, y, z)` cordinates of key points measured in mm. There are currently only 4 points as initially we are only trying to orientate the mouse skulls in space.---To match these to the points on an our images we need to find out how large the voxels (3D pixles) are:<jupyter_code>print("The voxel size is:", img.header.get_zooms(), 'mm in each dimension')<jupyter_output>The voxel size is: (0.035, 0.035, 0.035) mm in each dimension<jupyter_text>Therefore, we can divide the point location by the voxel size to get the points in space of the key points for this image.<jupyter_code>pixel_loc = np.round(tags / 0.035)
pixel_loc
data<jupyter_output><empty_output><jupyter_text>When plotted on the skull image we can see that these points pertain to the left and right eyes, left and right front molars and the tip of the nose. These are used to orientate the skull in 3D space in order to make labeling easier.<jupyter_code>def mri_point_plot(img, df, dim_cols=['x', 'z'], iter_cols='y'):
    """Graphs an points. pt_cols is used to set the cols to iterate 
    over (different views)
    """
    
    ax = []
    fig = plt.figure(figsize=(9, 8))
    columns = 3
    rows = 2

    for i in df.index:
        y_slice = int(df.loc[i, iter_cols])
        im = img[:, y_slice, :]
        ax.append( fig.add_subplot(rows, columns, i+1))
        ax[-1].set_title("Image depth: "+str(y_slice))  # set title
        plt.imshow(im)
        plt.plot(df.loc[i, dim_cols[0]], df.loc[i, dim_cols[1]], 'ro')
    plt.show()<jupyter_output><empty_output><jupyter_text>Another example of a skull:<jupyter_code>img2 = nib.load("/Users/michaeldac/Code/CUNY/698/Skulls/930.mnc")
tags2 = tag_parser("/Users/michaeldac/Code/CUNY/698/Skulls/930_landmarks.tag")



pix_size = img2.header.get_zooms()
print(pix_size)
img2 = img2.get_data()


tags2 = tags2 / pix_size[0]

mri_point_plot(img2, tags2)
img2 = nib.load("MouseSkullData/943.mnc")
tags2 = tag_parser("MouseSkullData/943_landmarks.tag")



pix_size = img2.header.get_zooms()
print(pix_size)
img2 = img2.get_data()


tags2 = tags2 / pix_size[0]

mri_point_plot(img2, tags2)
img2 = nib.load("/Users/michaeldac/Code/CUNY/698/Skulls/1837.mnc")
tags2 = tag_parser("/Users/michaeldac/Code/CUNY/698/Skulls/1837_landmarks.tag")



pix_size = img2.header.get_zooms()
print(pix_size)
img2 = img2.get_data()


tags2 = tags2 / pix_size[0]

mri_point_plot(img2, tags2)<jupyter_output><empty_output><jupyter_text>Explor image sizeThe actual image data when stored as a numpy array is huge at around 1 Gb<jupyter_code>import sys
sys.getsizeof(img_data)
print(round(sys.getsizeof(img_data) / 1e9, 2), "Gb")<jupyter_output><empty_output><jupyter_text>Further, we need to be concerned at the dimensions of the images and the voxel size. The image dimensions are important because many deep learning algorithms require a uniform image input size. Further we will most likely have to scale the images down in order to be abel to perform and not overfit on such highly dimensional data. The voxel size is also important because our scales are denoted in milimeters and we need to match them to the appropritate location even with scaling.<jupyter_code>import os
from tqdm import tqdm

files = os.listdir('/Users/michaeldac/Code/CUNY/698/Skulls')
mnc_files = [f for f in files if 'mnc' in f]
img_dims = {}
for i in tqdm(mnc_files):
    dims = nib.load(str('/Users/michaeldac/Code/CUNY/698/Skulls/' + i)).header.get_data_shape()
    img_dims[i] = dims

dim_df = pd.DataFrame.from_dict(img_dims).T
dim_df.columns = ['x', 'y', 'z']
dim_df.head()
img_res = {}
for i in tqdm(mnc_files):
    res = nib.load(str('/Users/michaeldac/Code/CUNY/698/Skulls/' + i)).header.get_zooms()
    img_res[i] = res
    
res_df = pd.DataFrame.from_dict(img_res).T
res_df.columns = ['x', 'y', 'z']
res_df.head()
res_df.loc[res_df.x != 0.035]
dim_df.describe()
dim_df.loc[dim_df.y == 888]<jupyter_output><empty_output><jupyter_text>So we can see that the voxel size is almost always `0.035` however there are some images that differ. Further outside of this intial training example we can expect the voxel sizes to differ considerably. Thus we need a solution to scale to whatever size is inputted. ----We also need to pick an image ratio to pad our images to. The issue is that the dimensions are not all even xor odd. This means that adding a uniform band around one side of an image will not be an option. Instead the image band or pad size will have to be different by one pixel in approximately half of the specimens.<jupyter_code>from ThreeDLabeler import images

from ThreeDLabeler.preprocessing import tag_parser
from ThreeDLabeler.plotting import mri_point_plot
# importlib.reload(ThreeDLabeler.images)



from preprocessing import mri_point_plot as mpp
from preprocessing import tag_parser
from preprocessing import Image
im = Image(data, (0.035, 0.035, 0.035), tags)


mpp(im.voxels, im.point_positon)
im.cube()
mri_point_plot(im.voxels, im.point_positon)
im.voxels
im.scale(128)
mri_point_plot(im.voxels, im.point_positon)
reduced_475 = (im.voxels, tags)
np.save('475_reduced.npy', reduced_475)
import os
os.getcwd()
reload_475 = np.load('475_reduced.npy', allow_pickle=True)
reload_475
print(im.point_positon)
print(im.voxels.shape)
mri_point_plot(im.voxels, im.point_positon)
im.cube()
print(im.point_positon)
print(im.voxels.shape)
im.scale(128)
type(im)
print(im.point_positon)
print(im.voxels.shape)
mri_point_plot(im.voxels, im.point_positon)<jupyter_output><empty_output><jupyter_text>We can see this is positioning<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline
from nilearn import plotting
plotting.plot_glass_brain("MouseSkullData/test.nii")   
import numpy as np
from scipy import ndimage
import matplotlib.pyplot as plt


class Image:
    """
    Processor class for annotating 3D scans.
    Arguments: 
    voxels: a 3D numpy array
    voxel_size: a tuple/list of three numbers indicating the voxel size in mm, cm etc
    point_position: the position in 3D of each point of interest. See tag_parser for more info
    """
    def __init__(self, voxels, voxel_size, point_position):
        self.voxels = voxels
        self.voxel_size = voxel_size
        self.point_position = point_position / voxel_size


    def cube(self):
        """Returns a cube image with all dimensions equal to the longest."""

        dims = self.voxels.shape
        max_dim = max(dims)
        
        x_target = (max_dim - dims[0]) / 2
        y_target = (max_dim - dims[1]) / 2
        z_target = (max_dim - dims[2]) / 2

        self.voxels = np.pad(self.voxels,
                            ((int(np.ceil(x_target)), int(np.floor(x_target))),
                             (int(np.ceil(y_target)), int(np.floor(y_target))),
                             (int(np.ceil(z_target)), int(np.floor(z_target)))),
                            'constant',
                            constant_values=(0))

        self.point_position = self.point_position + [np.ceil(z_target),
                                                     np.ceil(y_target),
                                                     np.ceil(x_target)]

        return(self)
        
    def scale(self, size=128):
        """
        Scales an cubic image to a certain number of voxels.
        This function relies on numpy's ndimage.zoom function
        """
        scale_factor = size / max(self.voxels.shape)
        self.voxels = ndimage.zoom(self.voxels, scale_factor)
        self.point_position = self.point_position * scale_factor
        self.voxel_size = False # To ignore this
        
        return(self)
import numpy as np
from tqdm import tqdm
from io import StringIO
import time
import os


def package_to_npy(file_path: str, mnc_files: list, tag_files: list, mnc_names: list):
    """
    INPUT:  Path where raw image files exist,
            List of .mnc files, 
            List of corresponding .tag files, 
            List of .mnc prefix names
    
    The .mnc file is loaded 
    The .tag file is parsed and converted to an ndarray via tag_parser()
    Processor class is instantiated with the .mnc and .tag file and cubes
    any images shaped as rectangular prisms and scales down image 
    resolution to 128x128x128.
    
    OUTPUT: Tuple of the processed .mnc and .tag files stored as .npy file 
    and saved to disk locally.
    """
    print('Starting image processing...')
    count = 0
    for i in tqdm(range(len(mnc_files))):
        img = nib.load(f'{file_path}/{mnc_files[i]}')
        tag = tag_parser(f'{file_path}/{tag_files[i]}')
        im = Processor(img.get_data(), img.header.get_zooms(), tag)
        im.cube().scale(128)
        npy_file = (im.voxels, im.point_position)
        np.save(f'{file_path}/{mnc_names[i]}.npy', npy_file)
        count += 1
    
    print(f'{count} .mnc/.tag file pairs have been processed and saved as .npy files')
x = reload[0]
y = reload[1]
y
img475 = Image(x, 1, y)
img475.cube()
img475.voxels.min()
nyp_cubed = (img475.voxels, img475.point_position)
np.save('/Users/michaeldac/Code/CUNY/698/Downloaded_Skulls/nyp_cubed.npy',nyp_cubed)

reloaded_nyp_cubed = np.load('/Users/michaeldac/Code/CUNY/698/Downloaded_Skulls/nyp_cubed.npy', allow_pickle=True)
reloaded_nyp_cubed[0].max<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1084923885325709, 'fraction_numerical': 0.042964222239394174, 'mean_word_length': 3.5078369905956115, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '137685', 'license': 'MIT', 'n_tokens_mistral': 4972, 'n_tokens_neox': 4512, 'n_words': 1347, 'path': 'eda.ipynb', 'repo_name': '3DLables/3DLabeler', 'size': '12941'}