AnatomyMR: A Multi-User Mixed Reality Platform for Medical Education

Mixed Reality (MR) has opened up new means of imparting education, especially in the medical field. Students could view the whole human anatomy and it’s components at multiple levels of scale to learn their location and naming nomenclature, and also interact with them separately to familiarize themselves with their functions. In this paper we propose a platform for Medical education in MR which creates shared experiences, providing support for teaching within classrooms. The application would also work as a standalone experience, augmenting important information over the anatomy model for the student’s reference when a shared experience is not in session, thereby facilitating self-study. The platform would be built in such a way that the instructors would also have the option to add 3D models and relevant content to the existing courseware and have it rendered in the mixed reality space.


INTRODUCTION
Anatomical education is the foundation of medical science, without a proper grasp of the functions and structure of the organs and biological systems a medical professional wouldn't be able to understand the effects a disease or accident might have on the human body. Of all the conventional approaches to imparting anatomical education cadaver dissection is considered to be the gold standard [1], but this approach has proven to be costly, time intensive, and found to induce a significant amount of psychological stress among medical students [2,3], furthermore, some students might not be able to learn through cadaver dissection due to cultural or religious beliefs [4]. With the advances in visualisation technologies and techniques,3D representations of the organs and biological systems can be displayed in immersive media such as Augmented Reality(AR), Mixed Reality(MR) and Virtual Reality(VR), to supplement the use of cadavers in teaching anatomy or maybe even completely replace the use of cadavers [5].
Mixed Reality(MR) is a hybrid between a digital reality and the real-world, the simulated 3d objects of the digital world are placed over the real-world environment in such a way that they are perceived to be a real-world object that interacts naturally with its environment. .

Fig.1: Mixed Reality
The Reality-Virtuality Continuum proposed by [6] shows Mixed Reality(MR) encompassing the spectrum of the different types of realities between Real Reality and Virtual Reality.
Mixed Reality could prove to be a powerful tool as an anatomical education tool, since the 3d models simulated in this immersive environment mimic real-world models, a model of the human anatomy could be displayed in MR with multiple levels of details and overlaid information. Also, MR based

EXISTING METHODOLOGIES
A number of MR/AR based education tools have been put forth for anatomical education or for specific medical procedures. [8] suggests an augmented reality enabled studio which used projection mapping to superimpose a 3d rendering of the anatomy over a person. This approach creates a collaborative environment for learning, as multiple people can perceive the projections without the aid of any special Head Mounted Display (HMD), but the need for a volunteer to act as a screen for the projections might not be ideal to teach the more complex systems and parts of the gross anatomy, since the projections might not perfectly conform to the volunteer's shape and posture, the rendering of the different parts of the anatomy might be disproportionate compared to the volunteer's actual anatomy. [9] is an augmented reality-based solution meant to be deployed in mobile applications, fiducial markers are used to augment the 3d models and the option to manipulate the rendered models are provided. Though this system might be better than the conventional methods of teaching [10] it's been found to be less effective than cadaver dissection [11], which might be due to the lack of immersion compared to a HMD based mixed experience where the 3d models might resemble and react to the environment like a physical object.

PROPOSED SYSTEM
We propose a multi-user mixed reality platform which creates a collaborative learning environment within classrooms. The system would ideally consist of multiple Mixed Reality HMDs with spatial mapping capabilities (such a the Microsoft Hololens) communicating over a network to render the 3d models of the anatomy at the same position and orientation with respect to the environment they are in, instead of using the HMD as a frame of reference to position the models, this would be executed through the sharing of spatial anchors [12]. The HMDs would also be aware of each other's position and direction of gaze in the room for a better collaborative environment.
The system would keep track of the type of user i.e student or faculty and generate a separate session to which the devices could be connected to, who is using the device to incorporate the dynamics that exist within classrooms, and thereby work as an effective supplementary tool for teaching. The system would also be provided with a desktop application which can be used to upload 3D Models and related information to a server, from which the HMDs would download the data to be displayed during class so that the application's contents aren't static and can be updated based on the courseware.

METHODOLOGY
The platform's functionality is dependent on the mode it's operated in : • Standalone Mode: This mode is used when the HMD is not connected to a network or when there's no other device in the network that is in the same session. Standalone mode provides a self-learning experience to the user, they are allowed to interact with and manipulate the parts of the anatomy model while being able to derive information from them through voice-overs triggered through interaction or through spatial annotations.
• Classroom Mode: This mode would be used when multiple devices are present in a session and a faculty's device is detected, the model displayed in this mode would have less integrated voiceovers and annotations compared to standalone mode, since the faculty is expected to guide the students through the experience.
When operated under classroom mode, to capture the dynamics that exist between the faculty and students, the application in the faculty's HMD would be equipped with extra functionalities such as: • Ability to manipulate and select models within the whole anatomy model • The ability to draw in 3d space During the session, the student's HMD would have the option to take screenshots or record certain parts and save as a video for a later session.

APPLICATIONS
This platform could be extended to teach other and more specialized medical topics, such as Histology and Neuroscience, provided the appropriate 3d representations of the cells or neural pathways are available. The effect of a disease on multiple levels of scale could be visualized more clearly, right from its effects on the gross anatomy and it's symptoms to the structure and working of the microbe that's causing the disease. Furthermore, real DICOM data of patients with certain diseases or patients who have met with an accident could be converted to 3D models and displayed within the application for a deeper understanding [13], with a streamlined workflow this approach could also be used for surgery planning as well [14].

CONCLUSION AND FUTURE WORKS
Mixed reality tools could revolutionize the medical industry especially medical education. With the improvements in processing power and MR hardware we can expect more and better applications that leverage this media to seep into everyday use for medical professionals. Thus, digital simulations of the human body could one day be realistic enough to completely replace the need for the use of cadavers to teach anatomy or practice surgical procedures. This platform is at its preliminary stages and requires more research to understand it's efficacy and to improve the interactions. With the everchanging MR hardware landscape, the platform would have to be standardized and scalable to make it device agnostic.