<commit_before>%%% The ``\documentclass'' command has one parameter, based on the kind of
%%% document you are preparing.
%%%
%%% [annual] - Technical paper accepted for presentation at the ACM SIGGRAPH 
%%%   or SIGGRAPH Asia annual conference.
%%% [sponsored] - Short or full-length technical paper accepted for 
%%%   presentation at an event sponsored by ACM SIGGRAPH
%%%   (but not the annual conference Technical Papers program).
%%% [abstract] - A one-page abstract of your accepted content
%%%   (Technical Sketches, Posters, Emerging Technologies, etc.). 
%%%   Content greater than one page in length should use the "[sponsored]"
%%%   parameter.
%%% [preprint] - A preprint version of your final content.
%%% [review] - A technical paper submitted for review. Includes line
%%%   numbers and anonymization of author and affiliation information.

\documentclass[annual]{acmsiggraph}

%%% If you are submitting your paper to one of our annual conferences - the 
%%% ACM SIGGRAPH conference held in North America, or the SIGGRAPH Asia 
%%% conference held in Southeast Asia - there are several commands you should 
%%% consider using in the preparation of your document.

%%% 1. ``\TOGonlineID''
%%% When you submit your paper for review, please use the ``\TOGonlineID''
%%% command to include the online ID value assigned to your paper by the
%%% submission management system. Replace '45678' with the value you were
%%% assigned.

\TOGonlineid{}

%%% 2. ``\TOGvolume'' and ``\TOGnumber''
%%% If you are preparing a preprint of your accepted paper, and your paper
%%% will be published in an issue of the ACM ``Transactions on Graphics''
%%% journal, replace the ``0'' values in the commands below with the correct
%%% volume and number values for that issue - you'll get them before your
%%% final paper is due.

\TOGvolume{0}
\TOGnumber{0}

%%% 3. ``TOGarticleDOI''
%%% The ``TOGarticleDOI'' command accepts the DOI information provided to you
%%% during production, and which makes up the URLs which identifies the ACM
%%% article page and direct PDF link in the ACM Digital Library.
%%% Replace ``1111111.2222222'' with the values you are given.

\TOGarticleDOI{1111111.2222222}

%%% 4. ``\TOGprojectURL'', ``\TOGvideoURL'', ``\TOGdataURL'', ``\TOGcodeURL''
%%% If you would like to include links to personal repositories for auxiliary
%%% material related your research contribution, you may use one or more of
%%% these commands to define an appropriate URL. The ``\TOGlinkslist'' command
%%% found just before the first section of your document will add hyperlinked
%%% icons to your document, in addition to hyperlinked icons which point to
%%% the ACM Digital Library article page and the ACM Digital Library-held PDF.

\TOGprojectURL{}
\TOGvideoURL{}
\TOGdataURL{}
\TOGcodeURL{}

%%% Replace ``PAPER TEMPLATE TITLE'' with the title of your paper or abstract.

\title{Medipa: Visualizing Medical Scans in 3D on the Web}

%%% The ``\author{}'' command takes the names and affiliations of each of the
%%% authors of your paper or abstract. The ``\thanks{}'' command takes the
%%% contact information for each author.
%%% For multiple authors, separate each author's information by the ``\and''
%%% command.

\author{<NAME>\thanks{e-mail: <EMAIL>}\\ North Carolina State University %
\and <NAME>\thanks{e-mail: <EMAIL>}\\North Carolina State University}

%%% The ``pdfauthor'' command accepts the authors of the work,
%%% comma-delimited, and adds this information to the PDF metadata.

\pdfauthor{<NAME>, <NAME>}

%%% Keywords that describe your work. The ``\keywordlist'' command will print
%%% them out.

\keywords{Volume Rendering, WebGL, Medical Images}

%%% The ``\begin{document}'' command is the start of the document.

%%% If you have user-defined macros, you may include them here.

% example of a user-defined macro called ``remark.''
% \newcommand{\remark}[1]{\textcolor{red}{#1}}

\begin{document}

%%% A ``teaser'' image appears under the title and affiliation information,
%%% horizontally centered, and above the two columns of text. This is OPTIONAL.
%%% If you choose to have a ``teaser'' image, it needs to be placed between
%%% ``\begin{document}'' and ``\maketitle.''

%\teaser{
%   \includegraphics[height=1.5in]{images/sampleteaser}
%   \caption{Spring Training 2009, Peoria, AZ.}
%}

%%% The ``\maketitle'' command must appear after ``\begin{document}'' and,
%%% if you have one, after the definition of your ``teaser'' image, and
%%% before the first ``\section'' command.

%%% Your paper's abstract goes in its own section.

\teaser{
	\includegraphics[scale=0.45]{head.png}
	\caption{\label{fig:head}Brain rendered with Medipa}
}

\maketitle

\begin{abstract}
We introduce Medipa, an open-source system to render medical images on the web in 3D that allow multiple medical representatives to investigate, analyze and annotate a single image.  A user can  upload medical image scans, adjust the zoom level, select pixels and highlight them, slice the volume.  Further, to harness the distributed nature of the web and provide a tool for collaboration users can save configurations that consist of the aforementioned adjustable properties and add comments. The rendering algorithm is based on volume ray-tracing using shaders implemented in WebGL, Javascript, and HTML5.
\end{abstract}
%%% ACM Computing Review (CR) categories.
%%% See <http://www.acm.org/class/1998/> for details.
%%% The ``\CRcat'' command takes four arguments.

\begin{CRcatlist}
  \CRcat{I.3.7}{Computer Graphics}{Three-Dimensional Graphics and Realism}{Radiosity};
\end{CRcatlist}

%%% The ``\keywordlist'' command prints out the keywords.

\keywordlist

%%% The ``\TOGlinkslist'' command will insert hyperlinked icon(s) to your
%%% paper. This includes, at a minimum, hyperlinked icons to the ACM article
%%% page and the ACM Digital Library-held PDF. If you added URLs to
%%% ``\TOGprojectURL'' or the other, similar commands, they will be added to
%%% the list of icons.
%%% Note: this functionality only works for annual-conference papers.

\TOGlinkslist

%%% The ``\copyrightspace'' command 
%%% Do not remove this command.

\copyrightspace

%%% This is the first section of the body of your paper.

\section{Introduction}
The current state of the art for viewing medical image scans requires using a desktop application.  This means having the software installed on every computer a medical health practitioner wishes to view the scans on as well as ensuring that computer runs the required operating system and has the required packages installed.  Further, these systems are geared towards viewing and recording information based on scan analysis not on collaborative efforts.

As web browser technology has evolved, access to the underlying GPU and APIs that follow the OpenGL standard have been added for doing complex 2D and 3D rendering.  This allows for the creation of complex applications using the browser as a platform that used to require a full desktop application.  The web provides a platform that is operating-system agnostic and allows users to use their favorite browser for interaction.  Further, by serving the content up from a central location, users can go anywhere in the world and still have access to viewing the scans.  This opens up a world where scans can be viewed by a variety of people and crowd-sourcing of ideas can be shared.

The rise of social media interaction, and distributed interaction of users has begun to spill over in to every field.  Further, as the availability of information increases and more and more users take responsibility of their personal data, the desire for people to want to see their medically relevant health information will rise.  By harnessing the web, data that has never been available at a personal level or able to be collaborated on can begin to flourish.  The MedIPA system aims to take a first step in this direction by providing volume rendered medical scans uploaded by users that can be manipulated, saved and commented on in order to procure a dialog.  In this way, users can view their own medical scans, share it with others, or seek further opinions on aspects of the images.  The same goes for medical professionals.

The appearance of HTML5 enables high quality drawing, provides flexible controls, and enhances the visual appearance of the web interface. Volume rendering was one of the challenges that users wanted to see on the web with interaction. Volume rendering is to display 3D scalar field dataset to a 2D image, but the computation time and memory load are the overhead. In this study, we demonstrate volume rendering of medical images on the web using volume ray-casting with HTML5. We also allow the users to save the configuration to keep history of the rendered image and to share and collaborate the result with others. 

The rest of the paper is organized as follows: Section 2 provides a look at related works in the area of volume rendering, Section 3 talks about the concept, Section 4 covers details of the implementation, Section 5 discusses limitations, Section 6 discusses possible future work and Section 7 finishes with conclusions.


\section{Related Works}
The technique of volume rendering gained a lot of interest especially to investigate medical data. The process of volume rendering has several stages to produce the result image \cite{drebin:1988}. First, the input volume data set is converted to a set of material percentage volumes. Each element in the volume is called a voxel (volume element) which the value represents the percentage of the material in a region of space. After classifying the dataset, a linear ramp called a transfer function assigns the opacity and color values to every voxel in the volume. It allows viewing a certain part of the volume by selecting a certain range of opacity and color. Next, the volume is projected to the image plane by compositing the opacity and colors of the voxels. 

Previous researches attempted multiple methods to visualize volumes. Marching cubes \cite{lorensen:1987} provided an algorithm that creates polygonal representation of iso-surfaces from a three dimensional array of data. It uses a table of possible edge intersections of a cube which describes how a surface cuts the surface. For realism, it calculates the normalized gradient from the original data which helps to create the desired surface model. However, the drawback of the algorithm is the speed regarding to the great number of triangles and overhead by rotation. The amount of memory needed for the surface result is another problem. Also, there exists a hole problem which you see a hole of the result image when at least one cube face has an intersection point in each of its four edges.

Levoy\cite{levoy:1990} introduced an efficient way to use ray-tracing for volume rendering. It improves its performance by employing binary volumes of coherent regions and adaptively terminating ray-tracing at the user-selected opacity threshold. However, the method is not interactive.
For interactivity, splatting provided a way to perform volume rendering in parallel. It is a technique that splats every voxel to the viewing surface in back-to-front order as if throwing a snow ball to a wall \cite{westover:1991}. This method runs in parallel, thus reduces the rendering time, but produces respectively lower image qualities. 
Recently, interesting work on WebGL have been conducted with volume rendering \cite{anatomical:2011}. 


\section{Concept}
	The solution provided by the MedIPA system focused on a two phase approach.  The first phase takes medical scans in their binary input format and processes them.  This processing is designed to turn the binary formats into a usable format when rendered in the browser and produce reductions of the data to provide lower quality images for faster loading and ease of user interaction.  The second phase involves taking the raw processed 2-dimensional data and rendering it within the browser as a volume.  The second involved two different approaches due to the first approach taken becoming unfeasible at scale.  The next three sub-sections describe the data processing phase and both attempts at 3D rendering.

\subsection{Data Processing}

Most medical image scans come in a binary format that can not be directly ported to WebGL for rendering.  Given this limitation, a small data processing library was created taking advantage of SimpleITK.  SimpleITK is a library that can turn binary format medical scans into data representations that can be worked on within native code.  The layer would take in the binary medical scan format, process with SimpleITK and then turn the outputted pixel data array into a series of PNG's that represented each layer of the scan.  The subsequent PNG is saved down to the file system along with a JSON based file that contains metadata about the image.  This metadata included filenames, dimensions, configuration information and pixel histogram information.

Given that some systems may be limited in their graphical processing power, down sampling of the image resolution was added to the data processing layer.  Whenever an image is processed by the server, the data points are reduced 3 times and for each reduction a PNG is saved off and entry made into the manifest file.  The reductions occur by selecting an octet of the pixels and finding an average location and value.  This produces images that are 1/8, 1/64 and 1/512 the size and dimensions of the original image scan.

Down sampling the image provides two direct benefits that are related to the reliance on the client to handle rendering of the data.  The first benefit comes from being able to reduce the size of the image bundle being sent across the wire.  Low resolution images translate to small file sizes.  This in turn means that when the user first accesses the render page a small image can be sent quickly across for the user to start viewing and interacting automatically.  As time progresses, the system can fetch higher resolution images and swap them in behind the scenes for the user to allow continued interaction without interruption.  

The second benefit comes from the variability in client computer graphics processing power.  Attempting to render high resolution images on an under powered machine can not only place the application into an unusable state but also render the entire client machine unusable by the user as the CPU and GPU attempt to handle the large resolution.  Thus, having lower resolution images helps to allow users with under powered machines to still interact with the system.  Further, the client code monitors the frames-per-second (FPS) as each data resolution is brought in and will halt the quality increases if the code determines that the system would enter an unusable state.

\subsection{Marching Cubes}
	The marching cube approach displays isosurfaces with constant scalar values from a 3D volume dataset. Given the constant value as an input, the algorithm tests the corners of each voxel in the scalar field and generates classified cubes. For each cubes, desired surfaces are extracted from the data set. We first attempted to use marching cubes to draw medical images on the web.

	This algorithm was implemented in the browser by creating a set of parallel Web Workers that each processed a different portion of the image array data.  Originally, the workers would each ask the server for the data set which introduced a lot of network traffic and redundant memory allocation.  Further improvements loaded the data set once and passed it to the workers.  Each worker processed independently and sent a signal with the return data back to a centralized worker queue.  Once all workers had finished the rendering algorithm took the results and generated the iso-surfaces.  
	The test bed system had 4GB of memory, a 2.3GHZ dual-core, hyper-threaded processor using 8 Web Workers.  For small data-sets (less than 500KB), the algorithm finished and rendered within a reasonable amount of time (< 1 second).  As the data began to scale in size, the computation time for the Marching Cubes algorithm increased to an unusable level capping out at a 25 minute calculation time for a 2.3MB set which roughly equated to over 1 million points.  Further, it became evident that the Marching Cubes approach would not allow for users to treat the rendered volume as a slice-able entity.

\subsection{Rendering algorithm}
	Volume rendering displays 3D scalar field dataset to a 2D image. The two traditional ways are slice-based rendering and volume ray tracing. Our algorithm follows volume ray tracing that shoots rays through the volume and samples along the path by equal intervals. As the sampling points are advanced along the ray, it accumulates the opacity and RGB values of the sampling points. By the use of transfer function, it assigns the color and opacity values to each voxel in the volume and shows the corresponding emission and absorption of light for the sample point. The transfer function enables to observe a specific part of the volume given the range of the density value in the dataset. We allowed easier interaction with the visual controls to manipulate the range of the dataset as well as the color and opacity values on the client side. 
The advantages of ray tracing are optimization from empty space skipping, independence of the projection view, and easiness of development. In order to improve performance, the algorithm renders the position of front and back facing triangles to the textures instead of calculating the intersection between the ray and the bounding box around the volume. The front face texture provides the start position of the ray and the result of subtracting from back to front face texture provides the direction of the ray. Then, starting from the start position of each ray, the algorithm accumulates the color and opacity value of the sampling point in front-to-back direction in parallel. In addition, the process is optimized to stop earlier when the accumulated opacity reaches to the maximum value (i.e.,1.0) or when the advance of ray tracing reached the length of the ray. It is possible by the back face texture which contains the end position of the ray for each pixel. The code is shown below.



$vec3 raystart = v_PosLocal; \\
vec2 texcoords = (v_Position.xy / v_Position.w + vec2(1.0, 1.0)) / 2.0; \\
vec3 rayend = texture2D(backpos_tex, texcoords).xyz; \\
vec3 raydir = rayend - raystart; \\
vec3 deltadir = normalize(raydir) * stepsize; \\
float len = length(raydir); \\
float deltadirlen = length(deltadir); \\
vec3 raypos = raystart + steps * interstep * deltadir; \\<commit_msg>modified "<" to \textless<commit_after>%%% The ``\documentclass'' command has one parameter, based on the kind of
%%% document you are preparing.
%%%
%%% [annual] - Technical paper accepted for presentation at the ACM SIGGRAPH 
%%%   or SIGGRAPH Asia annual conference.
%%% [sponsored] - Short or full-length technical paper accepted for 
%%%   presentation at an event sponsored by ACM SIGGRAPH
%%%   (but not the annual conference Technical Papers program).
%%% [abstract] - A one-page abstract of your accepted content
%%%   (Technical Sketches, Posters, Emerging Technologies, etc.). 
%%%   Content greater than one page in length should use the "[sponsored]"
%%%   parameter.
%%% [preprint] - A preprint version of your final content.
%%% [review] - A technical paper submitted for review. Includes line
%%%   numbers and anonymization of author and affiliation information.

\documentclass[annual]{acmsiggraph}

%%% If you are submitting your paper to one of our annual conferences - the 
%%% ACM SIGGRAPH conference held in North America, or the SIGGRAPH Asia 
%%% conference held in Southeast Asia - there are several commands you should 
%%% consider using in the preparation of your document.

%%% 1. ``\TOGonlineID''
%%% When you submit your paper for review, please use the ``\TOGonlineID''
%%% command to include the online ID value assigned to your paper by the
%%% submission management system. Replace '45678' with the value you were
%%% assigned.

\TOGonlineid{}

%%% 2. ``\TOGvolume'' and ``\TOGnumber''
%%% If you are preparing a preprint of your accepted paper, and your paper
%%% will be published in an issue of the ACM ``Transactions on Graphics''
%%% journal, replace the ``0'' values in the commands below with the correct
%%% volume and number values for that issue - you'll get them before your
%%% final paper is due.

\TOGvolume{0}
\TOGnumber{0}

%%% 3. ``TOGarticleDOI''
%%% The ``TOGarticleDOI'' command accepts the DOI information provided to you
%%% during production, and which makes up the URLs which identifies the ACM
%%% article page and direct PDF link in the ACM Digital Library.
%%% Replace ``1111111.2222222'' with the values you are given.

\TOGarticleDOI{1111111.2222222}

%%% 4. ``\TOGprojectURL'', ``\TOGvideoURL'', ``\TOGdataURL'', ``\TOGcodeURL''
%%% If you would like to include links to personal repositories for auxiliary
%%% material related your research contribution, you may use one or more of
%%% these commands to define an appropriate URL. The ``\TOGlinkslist'' command
%%% found just before the first section of your document will add hyperlinked
%%% icons to your document, in addition to hyperlinked icons which point to
%%% the ACM Digital Library article page and the ACM Digital Library-held PDF.

\TOGprojectURL{}
\TOGvideoURL{}
\TOGdataURL{}
\TOGcodeURL{}

%%% Replace ``PAPER TEMPLATE TITLE'' with the title of your paper or abstract.

\title{Medipa: Visualizing Medical Scans in 3D on the Web}

%%% The ``\author{}'' command takes the names and affiliations of each of the
%%% authors of your paper or abstract. The ``\thanks{}'' command takes the
%%% contact information for each author.
%%% For multiple authors, separate each author's information by the ``\and''
%%% command.

\author{<NAME>\thanks{e-mail: <EMAIL>}\\ North Carolina State University %
\and <NAME>\thanks{e-mail: <EMAIL>}\\North Carolina State University}

%%% The ``pdfauthor'' command accepts the authors of the work,
%%% comma-delimited, and adds this information to the PDF metadata.

\pdfauthor{<NAME>, <NAME>}

%%% Keywords that describe your work. The ``\keywordlist'' command will print
%%% them out.

\keywords{Volume Rendering, WebGL, Medical Images}

%%% The ``\begin{document}'' command is the start of the document.

%%% If you have user-defined macros, you may include them here.

% example of a user-defined macro called ``remark.''
% \newcommand{\remark}[1]{\textcolor{red}{#1}}

\begin{document}

%%% A ``teaser'' image appears under the title and affiliation information,
%%% horizontally centered, and above the two columns of text. This is OPTIONAL.
%%% If you choose to have a ``teaser'' image, it needs to be placed between
%%% ``\begin{document}'' and ``\maketitle.''

%\teaser{
%   \includegraphics[height=1.5in]{images/sampleteaser}
%   \caption{Spring Training 2009, Peoria, AZ.}
%}

%%% The ``\maketitle'' command must appear after ``\begin{document}'' and,
%%% if you have one, after the definition of your ``teaser'' image, and
%%% before the first ``\section'' command.

%%% Your paper's abstract goes in its own section.

\teaser{
	\includegraphics[scale=0.45]{head.png}
	\caption{\label{fig:head}Brain rendered with Medipa}
}

\maketitle

\begin{abstract}
We introduce Medipa, an open-source system to render medical images on the web in 3D that allow multiple medical representatives to investigate, analyze and annotate a single image.  A user can  upload medical image scans, adjust the zoom level, select pixels and highlight them, slice the volume.  Further, to harness the distributed nature of the web and provide a tool for collaboration users can save configurations that consist of the aforementioned adjustable properties and add comments. The rendering algorithm is based on volume ray-tracing using shaders implemented in WebGL, Javascript, and HTML5.
\end{abstract}
%%% ACM Computing Review (CR) categories.
%%% See <http://www.acm.org/class/1998/> for details.
%%% The ``\CRcat'' command takes four arguments.

\begin{CRcatlist}
  \CRcat{I.3.7}{Computer Graphics}{Three-Dimensional Graphics and Realism}{Radiosity};
\end{CRcatlist}

%%% The ``\keywordlist'' command prints out the keywords.

\keywordlist

%%% The ``\TOGlinkslist'' command will insert hyperlinked icon(s) to your
%%% paper. This includes, at a minimum, hyperlinked icons to the ACM article
%%% page and the ACM Digital Library-held PDF. If you added URLs to
%%% ``\TOGprojectURL'' or the other, similar commands, they will be added to
%%% the list of icons.
%%% Note: this functionality only works for annual-conference papers.

\TOGlinkslist

%%% The ``\copyrightspace'' command 
%%% Do not remove this command.

\copyrightspace

%%% This is the first section of the body of your paper.

\section{Introduction}
The current state of the art for viewing medical image scans requires using a desktop application.  This means having the software installed on every computer a medical health practitioner wishes to view the scans on as well as ensuring that computer runs the required operating system and has the required packages installed.  Further, these systems are geared towards viewing and recording information based on scan analysis not on collaborative efforts.

As web browser technology has evolved, access to the underlying GPU and APIs that follow the OpenGL standard have been added for doing complex 2D and 3D rendering.  This allows for the creation of complex applications using the browser as a platform that used to require a full desktop application.  The web provides a platform that is operating-system agnostic and allows users to use their favorite browser for interaction.  Further, by serving the content up from a central location, users can go anywhere in the world and still have access to viewing the scans.  This opens up a world where scans can be viewed by a variety of people and crowd-sourcing of ideas can be shared.

The rise of social media interaction, and distributed interaction of users has begun to spill over in to every field.  Further, as the availability of information increases and more and more users take responsibility of their personal data, the desire for people to want to see their medically relevant health information will rise.  By harnessing the web, data that has never been available at a personal level or able to be collaborated on can begin to flourish.  The MedIPA system aims to take a first step in this direction by providing volume rendered medical scans uploaded by users that can be manipulated, saved and commented on in order to procure a dialog.  In this way, users can view their own medical scans, share it with others, or seek further opinions on aspects of the images.  The same goes for medical professionals.

The appearance of HTML5 enables high quality drawing, provides flexible controls, and enhances the visual appearance of the web interface. Volume rendering was one of the challenges that users wanted to see on the web with interaction. Volume rendering is to display 3D scalar field dataset to a 2D image, but the computation time and memory load are the overhead. In this study, we demonstrate volume rendering of medical images on the web using volume ray-casting with HTML5. We also allow the users to save the configuration to keep history of the rendered image and to share and collaborate the result with others. 

The rest of the paper is organized as follows: Section 2 provides a look at related works in the area of volume rendering, Section 3 talks about the concept, Section 4 covers details of the implementation, Section 5 discusses limitations, Section 6 discusses possible future work and Section 7 finishes with conclusions.


\section{Related Works}
The technique of volume rendering gained a lot of interest especially to investigate medical data. The process of volume rendering has several stages to produce the result image \cite{drebin:1988}. First, the input volume data set is converted to a set of material percentage volumes. Each element in the volume is called a voxel (volume element) which the value represents the percentage of the material in a region of space. After classifying the dataset, a linear ramp called a transfer function assigns the opacity and color values to every voxel in the volume. It allows viewing a certain part of the volume by selecting a certain range of opacity and color. Next, the volume is projected to the image plane by compositing the opacity and colors of the voxels. 

Previous researches attempted multiple methods to visualize volumes. Marching cubes \cite{lorensen:1987} provided an algorithm that creates polygonal representation of iso-surfaces from a three dimensional array of data. It uses a table of possible edge intersections of a cube which describes how a surface cuts the surface. For realism, it calculates the normalized gradient from the original data which helps to create the desired surface model. However, the drawback of the algorithm is the speed regarding to the great number of triangles and overhead by rotation. The amount of memory needed for the surface result is another problem. Also, there exists a hole problem which you see a hole of the result image when at least one cube face has an intersection point in each of its four edges.

Levoy\cite{levoy:1990} introduced an efficient way to use ray-tracing for volume rendering. It improves its performance by employing binary volumes of coherent regions and adaptively terminating ray-tracing at the user-selected opacity threshold. However, the method is not interactive.
For interactivity, splatting provided a way to perform volume rendering in parallel. It is a technique that splats every voxel to the viewing surface in back-to-front order as if throwing a snow ball to a wall \cite{westover:1991}. This method runs in parallel, thus reduces the rendering time, but produces respectively lower image qualities. 
Recently, interesting work on WebGL have been conducted with volume rendering \cite{anatomical:2011}. 


\section{Concept}
	The solution provided by the MedIPA system focused on a two phase approach.  The first phase takes medical scans in their binary input format and processes them.  This processing is designed to turn the binary formats into a usable format when rendered in the browser and produce reductions of the data to provide lower quality images for faster loading and ease of user interaction.  The second phase involves taking the raw processed 2-dimensional data and rendering it within the browser as a volume.  The second involved two different approaches due to the first approach taken becoming unfeasible at scale.  The next three sub-sections describe the data processing phase and both attempts at 3D rendering.

\subsection{Data Processing}

Most medical image scans come in a binary format that can not be directly ported to WebGL for rendering.  Given this limitation, a small data processing library was created taking advantage of SimpleITK.  SimpleITK is a library that can turn binary format medical scans into data representations that can be worked on within native code.  The layer would take in the binary medical scan format, process with SimpleITK and then turn the outputted pixel data array into a series of PNG's that represented each layer of the scan.  The subsequent PNG is saved down to the file system along with a JSON based file that contains metadata about the image.  This metadata included filenames, dimensions, configuration information and pixel histogram information.

Given that some systems may be limited in their graphical processing power, down sampling of the image resolution was added to the data processing layer.  Whenever an image is processed by the server, the data points are reduced 3 times and for each reduction a PNG is saved off and entry made into the manifest file.  The reductions occur by selecting an octet of the pixels and finding an average location and value.  This produces images that are 1/8, 1/64 and 1/512 the size and dimensions of the original image scan.

Down sampling the image provides two direct benefits that are related to the reliance on the client to handle rendering of the data.  The first benefit comes from being able to reduce the size of the image bundle being sent across the wire.  Low resolution images translate to small file sizes.  This in turn means that when the user first accesses the render page a small image can be sent quickly across for the user to start viewing and interacting automatically.  As time progresses, the system can fetch higher resolution images and swap them in behind the scenes for the user to allow continued interaction without interruption.  

The second benefit comes from the variability in client computer graphics processing power.  Attempting to render high resolution images on an under powered machine can not only place the application into an unusable state but also render the entire client machine unusable by the user as the CPU and GPU attempt to handle the large resolution.  Thus, having lower resolution images helps to allow users with under powered machines to still interact with the system.  Further, the client code monitors the frames-per-second (FPS) as each data resolution is brought in and will halt the quality increases if the code determines that the system would enter an unusable state.

\subsection{Marching Cubes}
	The marching cube approach displays isosurfaces with constant scalar values from a 3D volume dataset. Given the constant value as an input, the algorithm tests the corners of each voxel in the scalar field and generates classified cubes. For each cubes, desired surfaces are extracted from the data set. We first attempted to use marching cubes to draw medical images on the web.

	This algorithm was implemented in the browser by creating a set of parallel Web Workers that each processed a different portion of the image array data.  Originally, the workers would each ask the server for the data set which introduced a lot of network traffic and redundant memory allocation.  Further improvements loaded the data set once and passed it to the workers.  Each worker processed independently and sent a signal with the return data back to a centralized worker queue.  Once all workers had finished the rendering algorithm took the results and generated the iso-surfaces.  
	The test bed system had 4GB of memory, a 2.3GHZ dual-core, hyper-threaded processor using 8 Web Workers.  For small data-sets (less than 500KB), the algorithm finished and rendered within a reasonable amount of time (\textless 1 second).  As the data began to scale in size, the computation time for the Marching Cubes algorithm increased to an unusable level capping out at a 25 minute calculation time for a 2.3MB set which roughly equated to over 1 million points.  Further, it became evident that the Marching Cubes approach would not allow for users to treat the rendered volume as a slice-able entity.

\subsection{Rendering algorithm}
	Volume rendering displays 3D scalar field dataset to a 2D image. The two traditional ways are slice-based rendering and volume ray tracing. Our algorithm follows volume ray tracing that shoots rays through the volume and samples along the path by equal intervals. As the sampling points are advanced along the ray, it accumulates the opacity and RGB values of the sampling points. By the use of transfer function, it assigns the color and opacity values to each voxel in the volume and shows the corresponding emission and absorption of light for the sample point. The transfer function enables to observe a specific part of the volume given the range of the density value in the dataset. We allowed easier interaction with the visual controls to manipulate the range of the dataset as well as the color and opacity values on the client side. 
The advantages of ray tracing are optimization from empty space skipping, independence of the projection view, and easiness of development. In order to improve performance, the algorithm renders the position of front and back facing triangles to the textures instead of calculating the intersection between the ray and the bounding box around the volume. The front face texture provides the start position of the ray and the result of subtracting from back to front face texture provides the direction of the ray. Then, starting from the start position of each ray, the algorithm accumulates the color and opacity value of the sampling point in front-to-back direction in parallel. In addition, the process is optimized to stop earlier when the accumulated opacity reaches to the maximum value (i.e.,1.0) or when the advance of ray tracing reached the length of the ray. It is possible by the back face texture which contains the end position of the ray for each pixel. The code is shown below.



$vec3 raystart = v_PosLocal; \\
vec2 texcoords = (v_Position.xy / v_Position.w + vec2(1.0, 1.0)) / 2.0; \\
vec3 rayend = texture2D(backpos_tex, texcoords).xyz; \\
vec3 raydir = rayend - raystart; \\
vec3 deltadir = normalize(raydir) * stepsize; \\
float len = length(raydir); \\
float deltadirlen = length(deltadir); \\
vec3 raypos = raystart + steps * interstep * deltadir; \\
{'fraction_non_alphanumeric': 0.0536887654255035, 'fraction_numerical': 0.007265345371013409, 'mean_word_length': 4.5829108261258575, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1623255', 'n_tokens_mistral': 9130, 'n_tokens_neox': 8598, 'n_words': 5571}