<jupyter_start><jupyter_text>Registering Multiple Slices to a Stack of Images __Purpose:__ Registering immunofluorescent whole brain slice images to a universal atlas and then saving the registered images into a stack.             NOTE: The notebook and package is currently developed for the workflow specific to the Nance Lab. __How the notebook works:__1. Accesses your Google Drive to pull files from2. Accesses the specific folder in Google drive with stored brain slice images3. Accesses your distances file for correlation of slices to an atlas4. Accesses the atlas segmentation file5. Loops through the files performing registration   * Downloads a file   * Cleans the files   * Obtains registration points for your image and atlas image   * Obtains registration coefficients for the transformation   * Performs the registration   * Saves the slice to a stack of images   * Deletes the file   * Moves to the next file and repeats steps6. Prints the stack of images for visual inspection __How to use the notebook__:  1. Store your slices in a Google Folder with the naming system 'slide_slice'2. Store your distances file in the same folder as 'distances.xlsx'3. Store your atlas in the same file under your desired name2. Input the Google Folder ID of neural slices (final part of the url) into the folder_id in Step 23. Input the name of the atlas in the atlas_file_name input of step 24. Run the whole notebook5. When a second tab opens, log into your Google Drive account6. Allow access to Quickstart    * DO NOT CLOSE THE TAB ONCE 'AUTHENTICATION FLOW IS COMPLETED' APPEARS7. Allow the notebook to run *Step 1: Importing Necessary Packages* For Google Drive Access: 1. pip install google-api-python-client2. pip install --upgrade google-api-python-client oauth2client3. pip install pydrive4. follow steps here: https://medium.com/@annissouames99/how-to-upload-files-automatically-to-drive-with-python-ee19bb13dda<jupyter_code>import os
from apiclient import discovery
from httplib2 import Http
import oauth2client
from oauth2client import file, client, tools
import pandas as pd
import nrrd
import nibabel as nib
import io
from googleapiclient.http import MediaIoBaseDownload

import numpy as np
from skimage.io import imread
import matplotlib.pyplot as plt

#Importing files from my packages
import ifmodels.register as register
import ifmodels.gdaccess as gdaccess
import ifmodels.preprocess as preprocess<jupyter_output><empty_output><jupyter_text>*Step 2: User inputs*<jupyter_code>folder_id = "'1NekZGpP0lg1CsUcKnw8zzaQzAnDgqsJc'"
atlas_file_name = 'atlas_segmentation.nrrd'<jupyter_output><empty_output><jupyter_text>*Step 3: Getting Google Drive Access*<jupyter_code>import pydrive
from pydrive.auth import GoogleAuth as gauth
from pydrive.drive import GoogleDrive

from pydrive.auth import GoogleAuth

#  Creates local webserver and auto handles authentication.
gauth = GoogleAuth()
gauth.LocalWebserverAuth()

#  Create GoogleDrive instance with authenticated GoogleAuth instance
drive = GoogleDrive(gauth)<jupyter_output>Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?client_id=311511771285-0thka75link8f72r0o48ktsmgjg5c756.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&access_type=offline&response_type=code

Authentication successful.<jupyter_text>*Step 4: Get list of file names and file ids from specified google folder*<jupyter_code>string = folder_id + ' in parents and trashed=false'
file_list = drive.ListFile({'q': string}).GetList()
names = []
ids = []
for file1 in file_list:
    names.append(file1['title'])
    ids.append(file1['id'])
file_names = pd.DataFrame(list(zip(names, ids)), columns = ['names','ids'])
file_names<jupyter_output><empty_output><jupyter_text>*Step 5: Accessing the distances file and reading into a dataframe*<jupyter_code>distances_file = file_names.loc[file_names['names'] == 'distances.xlsx']
distances_file = distances_file.reset_index()

gdistances = drive.CreateFile({'id': distances_file['ids'][0]})
gdistances.GetContentFile(distances_file['names'][0])

corr_atlas = pd.read_excel(distances_file['names'][0])

file_names = file_names[file_names.names!= 'distances.xlsx']<jupyter_output><empty_output><jupyter_text>*Step 6: Accessing the universal atlas*<jupyter_code>atlas_file = file_names.loc[file_names['names'] == atlas_file_name]
atlas_file = atlas_file.reset_index()

gatlas= drive.CreateFile({'id': atlas_file['ids'][0]})
gatlas.GetContentFile(atlas_file['names'][0])

F_im_nii = register.nrrd_to_nii(atlas_file_name)

file_names = file_names[file_names.names!= atlas_file_name]<jupyter_output><empty_output><jupyter_text>*Step 7: A inital test of files*<jupyter_code>file_id = str(file_names.at[4,'ids'])
new_file_name = str(file_names.at[4,'names'])
file6 = drive.CreateFile({'id': file_id})
file6.GetContentFile(new_file_name)
practice_im = register.im_read(new_file_name)<jupyter_output><empty_output><jupyter_text>*Step 7: Initializing a value and array for registration use in Step 8*<jupyter_code>Length = practice_im.shape[0] + 1500
Width = practice_im.shape[1] + 1500

#Creating a numpy stack to add the registered slices to 
Registered_stack = np.zeros((file_names.shape[0], Length, Width))
sagittal, coronal, horizontal = register.atlas_slice(F_im_nii, 1)
atlas_length = coronal.shape[0]
resize_value = int(practice_im.shape[0]/atlas_length)<jupyter_output><empty_output><jupyter_text>*Step 8: Full code for pulling and registering files from google drive*<jupyter_code>#  Cleaning up the moving image and getting registration points
im = register.im_read(new_file_name)
binary = register.mim_edge_detector(im)
binary = register.image_cleaning(binary)
coor_df = register.find_points(binary)

#  Getting the proper atlas slice an its registration points
slice = corr_atlas.loc[corr_atlas['Slidetag'] == new_file_name[:-4]].values.tolist()[0]
slice_number = slice[1]
sagittal, coronal, horizontal = register.atlas_slice(F_im_nii, slice_number)
resized = preprocess.resize(coronal, resize_value)
resized = register.mim_edge_detector(resized)
fim_coor_df = register.find_points(resized)
    
 #  Performing the affine transformation to get registration coordinates
df = pd.concat([coor_df, fim_coor_df], axis = 1)
ainv = register.reg_coefficients(df, 1, 4, 6)
    
#  Registering the slice
registered_im = register.registration(im, resized, ainv)
        
Registered_stack[1, 0:registered_im.shape[0], 0:0+registered_im.shape[1]] = registered_im

plt.imshow(binary, cmap = 'gray')
plt.title('Layer of Registered Stack')
plt.xticks([])
plt.yticks([])

plt.show()
k = 0
for row in file_names.itertuples():
    try:
        #  Downloading the slice to be registered (moving image)
        file_id = row.ids
        new_file_name = row.names
        file6 = drive.CreateFile({'id': file_id})
        file6.GetContentFile(new_file_name)

        #  Cleaning up the moving image and getting registration points
        im = register.im_read(new_file_name)
        binary = register.mim_edge_detector(im)
        binary = register.image_cleaning(binary)
        coor_df = register.find_points(binary)

        #  Getting the proper atlas slice an its registration points
        slice = corr_atlas.loc[corr_atlas['Slidetag'] == new_file_name[:-4]].values.tolist()[0]
        slice_number = slice[1]
        sagittal, coronal, horizontal = register.atlas_slice(F_im_nii, slice_number)
        resized = preprocess.resize(coronal, resize_value)
        resized = register.mim_edge_detector(resized)
        fim_coor_df = register.find_points(resized)
    
        #  Performing the affine transformation to get registration coordinates
        df = pd.concat([coor_df, fim_coor_df], axis = 1)
        ainv = register.reg_coefficients(df, 1, 4, 6)

        #  Registering the slice
        registered_im = register.registration(im, resized, ainv)
        
        Registered_stack[k, 0:registered_im.shape[0], 0:0+registered_im.shape[1]] = registered_im
    
        #  Removing the downloaded file and printing file name as a status update
        os.remove(new_file_name)
        k = k + 1
        print(k, new_file_name)
    
    except:
        print("error " + str(IOError) + new_file_name)<jupyter_output>1 slide4_slice3.tif<jupyter_text>*Step 9: Saving the created stack as a Nifti File*<jupyter_code>full_brain = nib.Nifti2Image(Registered_stack, affine=np.eye(4))<jupyter_output><empty_output><jupyter_text>*Visualization 1: Looking at an individual slice*<jupyter_code>plt.imshow(Registered_stack[4], cmap = 'gray')
plt.title('Layer of Registered Stack')
plt.xticks([])
plt.yticks([])

plt.show()<jupyter_output><empty_output><jupyter_text>*Visualization 2: Looking at the full stack of slices side by side*<jupyter_code>Registered_stack.shape
fig, ax = plt.subplots(1,5, figsize=[18,18])

n=0
slice=1
for _ in range(5):
    ax[n].imshow(Registered_stack[_], 'gray')
    ax[n].set_xticks([])
    ax[n].set_yticks([])
    ax[n].set_title('Slice number: {}'.format(slice), color='black')
    n += 1
    slice += 1
    
fig.subplots_adjust(wspace=0, hspace=0)
plt.show()<jupyter_output><empty_output><jupyter_text>Below this line is very developmental work *Working with matplotlib 3D*<jupyter_code>from mpl_toolkits import mplot3d
fig = plt.figure()
ax = plt.axes(projection = '3d')
Registered_stack.shape
Registered_stack[0].shape
Registered_stack[0][1]
#Data for three deminsional stuff
ax = plt.axes(projection = '3d')

# create a 21 x 21 vertex mesh
xx, yy = np.meshgrid(np.linspace(0,1,21), np.linspace(0,1,21))

# create vertices for a rotated mesh (3D rotation matrix)
X =  xx 
Y =  yy
Z =  10*np.ones(X.shape)

data = Registered_stack[3]

ax.plot_surface(X, Y, Z, rstride=1, cstride=1, facecolors=plt.cm.BrBG(data), shade=False)
from mayavi import mlab
mlab.init_notebook()
volume = mlab.pipeline.volume(mlab.pipeline.scalar_field(Registered_stack), vmin=0, vmax=0.8)

mlab.draw()
mlab.savefig('output.png')
mlab.figure()
mlab.contour3d(Registered_stack, contours=[])
mlab.outline(color=(0,0,0))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09968416896960126, 'fraction_numerical': 0.01914725621792341, 'mean_word_length': 5.454140127388535, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 51, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '476832', 'license': 'MIT', 'n_tokens_mistral': 3536, 'n_tokens_neox': 3382, 'n_words': 870, 'path': 'scripts/ifmodels.ipynb', 'repo_name': 'hhelmbre/ifmodels', 'size': '10132'}