Purpose The data which is available to surgeons before, during and after surgery is steadily increasing in quantity as well as diversity. When planning a patient’s treatment, this large amount of information can be difficult to interpret. To aid in processing the information, new methods need to be found to present multimodal patient data, ideally combining textual, imagery, temporal and 3D data in a holistic and context-aware system.
@cite_18 implemented a VR plug-in for the medical image processing software . The plug-in focuses on the rendering of the virtual scene and relies on MeVisLab for loading and management of the data. However, this method only moves small portions of the available data into VR, so relationships between the inhomogeneous data sources cannot be visualized and context awareness may be lost.
@cite_18
2602830123 2748157007 2604690721 2801778672
Virtual Reality, an immersive technology that replicates an environment via computer-simulated reality, gets a lot of attention in the entertainment industry. However, VR has also great potential in other areas, like the medical domain, Examples are intervention planning, training and simulation. This is especially of use in medical operations, where an aesthetic outcome is important, like for facial surgeries. Alas, importing medical data into Virtual Reality devices is not necessarily trivial, in particular, when a direct connection to a proprietary application is desired. Moreover, most researcher do not build their medical applications from scratch, but rather leverage platforms like MeVisLab, MITK, OsiriX or 3D Slicer. These platforms have in common that they use libraries like ITK and VTK, and provide a convenient graphical interface. However, ITK and VTK do not support Virtual Reality directly. In this study, the usage of a Virtual Reality device for medical data under the MeVisLab platform is presented. The OpenVR library is integrated into the MeVisLab platform, allowing a direct and uncomplicated usage of the head mounted display HTC Vive inside the MeVisLab platform. Medical data coming from other MeVisLab modules can directly be connected per drag-and-drop to the Virtual Reality module, rendering the data inside the HTC Vive for immersive virtual reality inspection. A large body of research is available on data visualization and many tools have been developed to visually comprehend large healthcare datasets. However, little attention has been paid to the area of visualization using virtual reality (VR). In order to fill this gap, we designed and developed a tool called V Rvisu to visualize large and complex medical datasets using VR. More specifically, our research focuses on creating 3D images of tumors from real-world medical datasets and displays them in a virtual reality environment in meaningful ways. The tool provides novel visualizations for medical practitioners to identify and analyze the change in shape of tumors. Introduction Virtual reality (VR) and head mount displays (HMDs) have been advanced for multimedia and information technologies but have scarcely been used in surgical training. Motion sickness and individual psychological changes have been associated with VR. The goal was to observe first experiences and performance scores using a new combined highly immersive virtual reality (IVR) laparoscopy setup. Real-time 3D scene reconstruction from RGB-D sensor data, as well as the exploration of such data in VR AR settings, has seen tremendous progress in recent years. The combination of both these components into telepresence systems, however, comes with significant technical challenges. All approaches proposed so far are extremely demanding on input and output devices, compute resources and transmission bandwidth, and they do not reach the level of immediacy required for applications such as remote collaboration. Here, we introduce what we believe is the first practical client-server system for real-time capture and many-user exploration of static 3D scenes. Our system is based on the observation that interactive frame rates are sufficient for capturing and reconstruction, and real-time performance is only required on the client site to achieve lag-free view updates when rendering the 3D model. Starting from this insight, we extend previous voxel block hashing frameworks by introducing a novel thread-safe GPU hash map data structure that is robust under massively concurrent retrieval, insertion and removal of entries on a thread level. We further propose a novel transmission scheme for volume data that is specifically targeted to Marching Cubes geometry reconstruction and enables a 90 reduction in bandwidth between server and exploration clients. The resulting system poses very moderate requirements on network bandwidth, latency and client-side computation, which enables it to rely entirely on consumer-grade hardware, including mobile devices. We demonstrate that our technique achieves state-of-the-art representation accuracy while providing, for any number of clients, an immersive and fluid lag-free viewing experience even during network outages.