Augmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go further and let them collaborate with the physical elements around them, including the objects that belong to the Internet of Things (IoT). Unfortunately, AR/MR and IoT devices usually make use of heterogeneous technologies that complicate their intercommunication. Moreover, the implementation of the intercommunication mechanisms requires involving specialized developers with have experience on the necessary technologies. To tackle such problems, this article proposes the use of a framework that makes it easy to integrate AR/MR and IoT devices, allowing them to communicate dynamically and in real time. The presented AR/MR-IoT framework makes use of standard and open-source protocols and tools like MQTT, HTTPS or Node-RED. After detailing the inner workings of the framework, it is illustrated its potential through a practical use case: a smart power socket that can be monitored and controlled through Microsoft HoloLens AR/MR glasses. The performance of such a practical use case is evaluated and it is demonstrated that the proposed framework, under normal operation conditions, enables to respond in less than 100 ms to interaction and data update requests. The Internet of Things (IoT) paradigm has already been considered for multiple applications in fields like smart appliances ,2, preciThe first pioneering AR/MR developments were carried out in the 1960s ,9, but tLuckily, during the last years AR/MR devices have improved significantly thanks to advances on electronics, computing technologies and wireless communications. Such advances have also led to decrease the AR/MR commercialization price, which has sparked the interest of new consumers and industries that are transforming their processes through the Industrial IoT (IIoT) or Industry 4.0 paradigms .One of the most relevant challenges that AR/MR developers currently face when interacting with IoT devices is the technology heterogeneity, which makes it difficult to implement even simple interactions. Thus, most AR/MR frameworks use technologies that differ remarkably to those used for the development of IoT device software. Moreover, the skills usually required by AR/MR and IoT experts are also different, what complicates the communication between both fields.In order to allow AR/MR applications to interact with the surrounding IoT devices, it is necessary to develop communication mechanisms that enable exchanging data in the same ’language’, so that the involved devices understand each other. Unfortunately, the development of such mechanisms is not straightforward due to the previously mentioned heterogeneity issues, the computational hardware limitations  and the development restrictions imposed by AR/MR frameworks.To tackle the aforementioned problems, this article includes the following contributions. First, it is faced the challenge of interconnecting very different types of systems by proposing an AR/MR-IoT framework that allows for integrating AR/MR platforms and IoT devices easily through the use of open-source standard communication protocols. Such a framework enables implementing ubiquitous and scalable applications in a flexible way and eases their configuration. In addition, the proposed framework is designed to allow for implementing on top of it complex functionality in a simple way. Such a development is eased by providing the framework software through GitHub together with implementation examples . MoreoveThe rest of this article is organized as follows. In the last years, novel smart environments have been conceived under what has been denominated as Extended Reality (XR). In such spaces, digital, physical, and social layers are strongly intertwined enabled by technologies such as AR/MR, Virtual Reality (VR) and immersive displays . Such teThe IoT is a field where protocol, standard and hardware compatibility problems are common due to the diversity of manufacturers and developers . The menRegarding AR/MR-IoT frameworks, four relevant works can be cited. The first one is described in , where tAdditionally, it should be noted that other works focused on facing the specific challenges that pose client-server infrastructures, IoT resource-constrained devices, mobility, interaction methods, data management or ad-hoc 3D streaming. One of such works is described in , where tUser eXperience (UX) is a key factor to consider when designing an AR/MR-IoT framework since UX will determine to a great extent the likelihood of user adoption. Quality of Experience (QoE) evaluation has been an active research topic for the past years in the AR/MR field. Academic works range from the QoE evaluation of basic AR systems ,35 to moOne of the most complete evaluations of an HMD was performed in . Such a Finally, with respect to explicit metrics, it was used a Likert scale questionnaire to subjectively report QoE under six variables: utility, usability, interaction, aesthetics, efficiency and acceptability. Emotional state was reported by participants using a Self-Assessment Manikin (SAM) questionnaire to reflect their emotions upon task realization. The SAM questionnaire involves three scales, one for each dimension of affect . The results highlight the potential of AR with respect to efficiency and productivity: AR yielded higher success rates, significantly shorter time to completion, more positive emotions and less stressed states than paper-based instructions. However, users remark some issues with respect to aesthetics, therefore acknowledging the importance of a human-centered AR design.Another work worth mentioning studied the user experience of a visual analytics system for the drifting data of bees . In suchTaking into account the results of the previous QoE works, it can be concluded that efficiency is one of the most important criteria regarding AR/MR user experience. Since other criteria for QoE assessment like effectiveness or satisfaction are strongly dependent on the AR/MR-IoT application, this work will only focus on efficiency, aiming to minimize task completion time by optimizing user interactions. Such an efficiency, when considering an AR/MR-IoT framework, is strongly correlated with throughput and latency values. Therefore, After reviewing the state of the art it can be concluded that there are only a few previous works with similar aims respect to the proposed AR/MR-IoT framework. However, in contrast to the work presented in this article, the vast majority of the previous works presented are very early developments with a relevant number of open issues that require further research.The main problem that the proposed framework aims to solve is to reduce the existing difficulties to interconnect heterogeneous systems such as those found in IoT and AR/MR devices. For such a purpose, this framework defines different mechanisms and methods to interconnect diverse technologies by using standardized protocols that can be easily integrated into both the target platforms and existing projects.This section describes the challenges of a AR/MR-IoT system, the requirement analysis of the system as well as the design of each layer of the framework and how communications are handled by each component of the system.The process of implementing an AR/MR-IoT framework involves different challenges that are often overlooked and that should be considered during the design phase. As it was previously mentioned, one of main challenges is the heterogeneity of the technologies that need to be used in order to build a complete AR/MR-IoT system. This usually involves making low-level decisions about the architecture and software on diverse areas of knowledge.Another important challenge is latency. In order to provide a proper user experience, it is necessary to keep response times low for the different communication protocols, component interactions and processing tasks. If a part of the system is related to delays in the information flow, a bottleneck will exist that will eventually affect the whole system, harming its usability.Both AR/MR and IoT systems are usually complex, which makes difficult to integrate a new communication architecture into the existing software. This means that the technologies used by an AR/MR-IoT framework to implement the different interactions should be able to be integrated easily into existing projects. This can be achieved by providing well-documented open-source software.Some of the works previously analyzed in Latency. This is the metric that makes it possible to quantify how fast the system can react to a user actions. According to the International Telecommunication Union (ITU), the following latency times should be considered when assessing visual interaction performance :Human reaction time , is roughly 1 s.Human auditory reaction time is about 100 ms.Human visual reaction time is in the range of 10 ms.An interface is said to have a fast reaction time if it is close to 1 ms.However, it is important to note that the previously indicated latencies are ideal approximations and in practice, when using AR/MR devices, they vary depending on the person, on the type of interaction and on the characteristics of the visualized content. For instance, the indicated human visual reaction time is aimed at providing a good experience when visualizing fast moving objects. In practice, most of the elements presented through an AR/MR interface do not move fast, so a good user experience can be provided despite the existence of latencies higher than 10 ms.Compatibility. The system must be able to interact with heterogeneous devices transparently. In addition, the data types required by an AR/MR interface are very different from those usually required by an IoT device, so an AR/MR-IoT framework must adapt the protocols to the needs of each device.Ease of development. An AR/MR-IoT framework implementation should be easy to replicate and flexible enough to make it easy to modify it to be adapted to new use cases. The upper layers of the framework should allow for such a flexibility. For instance, in the framework presented in this article, the use of a software like Node-RED makes it possible to change data flows graphically, by just changing nodes through a visual interface.Ease of integration. The framework should be easy to integrate within existent applications. As it was mentioned before, the characteristics of current AR/MR and IoT systems are very different, so the framework should be able to integrate every AR/MR and IoT seamlessly.As it was previously mentioned, one of the most important characteristics of an AR/MR application is the response time to the actions carried out by a user. If the response time is high, the user experience will not be good and the interaction would turn out to be less intuitive and pleasant. In addition, there are other requirements imposed by the type of devices used to implement the system as well as by the cost of implementation. Specifically, the main requirements for a AR/MR-IoT framework are the following:The AR/MR Device Sublayer is composed by the diverse AR/MR devices , which are able to exchange information with the AR/MR-IoT Layer through a wireless Access Point (AP). The pink arrows of The IoT Device Sublayer is formed by the different IoT networks managed by the system. Such networks are composed essentially by sensors and actuators, which may be part of other systems like smart appliances, industrial machinery or home automation systems. The green arrows of Regarding the AR/MR-IoT Layer, it is responsible for interconnecting AR/MR and IoT devices. It may be implemented on a remote cloud on the Internet or on a local edge computing server . DueIt may be tempting to try to implement the entire system using IoT communication protocols like Message Queuing Telemetry Transport (MQTT), but, unfortunately, they are currently not appropriate for implementing AR/MR-IoT applications: such applications often require sending large amounts of information  that are difficult to handle through MQTT and similar IoT protocols, which have been designed for managing small payloads. This would not fulfill the ease-of-use requirement, since designing a protocol for managing large amounts of data through the mentioned IoT protocols is usually complicated and not very efficient.Moreover, in some AR/MR development environments it is not easy to make use of protocols like MQTT due to the restrictions imposed by the available AR/MR frameworks and high-level APIs. As the ease-of-integration requirement indicates, the framework should be integrated easily, but the cost of implementing MQTT or similar protocols on top of proprietary APIs, if possible, would be higher than other alternatives already available for the target AR/MR framework. Nonetheless, according to the framework requirements, protocols like MQTT can be used only for communicating IoT devices, while other solutions need to be devised to communicate such devices with AR/MR platforms. Therefore, a mixed approach is required to fulfill all the framework requirements.REST Application Programming Interface (API). It is in charge of managing the data exchanges related to the AR/MR devices. Many AR/MR frameworks offer a very high-level programming API that make it difficult to access low-level communications functionality (in some cases it is even impossible). A REST API ensures compatibility with most of the existing AR/MR frameworks in a simple way and it is flexible enough to be added to existing projects with little effort, thus fulfilling the ease-of-integration requirement.MQTT broker. It allows for implementing a publish/subscribe service that enables IoT devices to send and to receive data asynchronously. It is based on MQTT, which is an open standard protocol that is really lightweight, so it can be easily implemented even in resource-constrained IoT devices. This protocol is widely used in heterogeneous IoT networks and it is supported by many IoT platforms. It also provides other appealing features for IoT networks like low-latency, low-power and delay tolerance.AR/MR-IoT Bridge Service. This component is essential for the whole framework, since it is the one responsible for actually interconnecting the different AR/MR and IoT components. As the ease-of-development requirement states, it is essential for this component to be easily configurable so that it can be changed to fit each application. In practice, this component delivers the collected IoT data to the AR/MR devices that request them and sends commands from such AR/MR devices to IoT devices. Apart from protocol translation, it also performs caching operations, storing all recent data values shared by the IoT devices. Thus, response times are reduced since data are always available when a user application requests them, even when the device is in low-power mode, temporarily busy or unavailable. Furthermore, the exchanged data flows are processed within this component so that each device receives the information in the most appropriate format (thus fulfilling the framework compatibility requirement).Considering the previous aspects, this article proposes a design that is essentially composed by the following software components:Surrounding IoT devices: they generate data on their sensors and internal state.AR/MR device sensors: they collect information from their embedded sensors, which allow for detecting where the user currently is, where he/she has previously been, what is he/she looking at or his/her posture.User profile: the framework can make use of relevant information on the user in order to later use it to determine the potential actions that he/she can carry out or the information he/she can access to. For example, in a smart home, it may be interesting to limit the actions that children can perform on certain appliances. Moreover, a user profile can be linked to past actions with the objective of using them as contextual information for future actions. For instance, the framework could infer the habits of a user from his/her past actions  and prepare the IoT devices with which the user interacts more frequently before the user requires their use (thus decreasing interaction latency and decreasing energy consumption).External inputs: certain external inputs, either from remote users or external services  or weather forecast services) can influence when and how certain actions are performed on IoT or AR/MR devices. For example, the interactions of an AR/MR user on an IoT device may be different if they are performed during daytime or at night.System configuration parameters: the administrator of the system can establish specific rules and thresholds that determine how AR/MR user interactions are carried out on IoT devices.With respect to the processing components shown in Finally, regarding the outputs illustrated in The proposed framework was designed in such a way that it allows for implementing on top of it complex functionality in a simple way. An example of a complex functionality is related to the nature of many IoT devices: they require to implement an energy consumption policy to provide an adequate compromise between response time and low-power consumption. The proposed AR/MR-IoT framework allows, for instance, to make use of the location information of IoT and AR/MR devices to warn the former when an AR/MR user is close, so that IoT devices are ready in case the user decides to interact with them. Location can be determined by using the surrounding WiFi access points to which the different devices are connected to. The gathered location information can be sent to the framework Bridge Service, which stores it and takes the corresponding actions when a user moves from one area to another.An IoT device can remain in a low-power consumption mode until it receives a notification that indicates that an AR/MR user is in the vicinity. Then, it changes its energy saving configuration and starts to make more frequent updates, so response times are significantly reduced. In addition, when a user moves away from an IoT device, such a device can be warned so as to return to its low-power consumption mode.User A sends a command to Device 1, which is not in a nearby area, so its response time is larger than when interacting with closer IoT devices. In contrast, User B sends a command to Device 3, which is in the vicinity, so its response time is reduced significantly.Moreover, it can be observed in The theoretical architecture detailed in The practical implementation of this sublayer takes into account that IoT devices are usually constrained in terms of computational power and that they have to be efficient in terms of energy when they rely on batteries. As a consequence, the communications protocols used by this sublayer have to be lightweight in relation to the amount of necessary computational resources and required power consumption . MoreoveThe previously mentioned necessary features justify the use of MQTT, which has been recently standardized , is lighAmong the diverse AR/MR hardware devices currently available ,46, the As of writing, Microsoft HoloLens smart glasses  provide Gaze Manager and Gaze Stabilizer modules: they track the user head orientation and where a user is looking at.Spatial Mapping module: it obtains and updates a 3D-map of the user surroundings.Gesture Manager module: it recognizes the hand gestures performed by the user. In The most relevant HoloLens SDK modules used by the implemented application are summarized in When an AR/MR device wants to perform an action on an IoT device , it sends an HTTP request to Node-RED through the REST API.The HTTP request is processed by Node-RED and then it decides to which IoT device it should be forwarded to. Such a decision is usually conditioned by a session token that is embedded on the AR/MR device request.The request is forwarded to the IoT device through an MQTT message that is published on Mosquitto under a specific topic.Since the target IoT device will be subscribed to the topic where the MQTT message was published, it will consume it as soon as such an IoT device receives the new message notification from Mosquitto.To send the data, the IoT device also makes use of MQTT messages through Mosquitto. Such messages are stored by Node-RED and then forwarded to the AR/MR devices that request for updates.This layer runs on a cloud an MQTT server called Mosquitto , Node-REAs it can be observed from the previous description, the core of the implementation is Node-RED, which enables interconnecting the different elements of the system. Node-RED is a visual-programming tool based on Node.js  that allIn order to validate the proposed design and to assess the level of compliance of the framework requirements, a practical application was analyzed and designed to provide a reference implementation. Such an application consists of a smart power outlet for monitoring energy consumption that can be controlled from a pair of Microsoft HoloLens smart glasses. Specifically, the proposed system provides AR/MR users with real-time data collected from a commercial smart power outlet. Such users can also control the switching on and off of the power outlet by interacting with a virtual dashboard through gestures.It must be noted that the proposed practical implementation was devised to provide fast response times so as to deliver a good user experience when using the AR/MR application. In addition, it was designed to be easily replicated and to asses easily the compatibility of IoT devices with the proposed framework. The next subsections detail the main components of the system.The used IoT smart power socket is a commercial device  that haThe smart socket can monitor the current that flows through the socket and log the intensity, so it can determine the instantaneous power and the consumed energy. In addition, the smart socket allows for sending ON and OFF commands through a standard MQTT communications channel. Such a channel is also used to send back the collected energy consumption values.The smart socket firmware is based on Tasmota , which iNode-RED is the technology responsible for the bridge service operations. It translates the information between the different protocols used to communicate with both the IoT and the AR/MR device sublayers. It also performs caching operations to improve the latency of the system and modifies data types to make the system interoperable. It was configured as it is shown in the screenshot in The connections between the nodes indicate the existing data flows while the nodes represent inputs and outputs as well as data processing nodes. The proposed setup makes it easy to reconfigure the data flows to fit new particular use cases, requiring little effort.The REST API is composed of different endpoints that allow applications to perform all the necessary actions. All of them are exposed through Node-RED HTTP service and are intended for reading and writing data from the applications. A brief description of each of them is given in For security reasons, all endpoints accept as input parameters the smart socket identifier and a cookie with an authentication token that identifies the user./cmnd: topics with this root are requests sent from a client to an IoT device./stat: topics with this root are always responses to cmnd-type messages./tele: topics with this root correspond to messages sent by the devices periodically to update their status.The MQTT message hierarchy was structured around topics divided into three levels. The first level was related to the root MQTT topics, which were defined depending on the type of message to be handled:token_hwId.The second topic level of the message hierarchy corresponds to the socket identifier, which includes the user token and the unique hardware identifier of the socket with the format /sensor: It is used for obtaining the consumption and power data of the socket./state: It is used for collecting the socket status information ./result: It is used to notify the result of the requests to the socket./timerX: It is used to exchange information and actions on the timers./power: It obtains the power status of the socket  or 0 (OFF))./timers: It enables sending together the information of all the 16 timers./prices: It is used to send the energy price per hour list.The last level of the hierarchy indicates an action or is related to certain information:Finally, it is worth pointing out that the messages exchanged through the REST API and MQTT are JSON files, which allow for standardizing the data types defined by the system.Users can interact with the dashboard by clicking on the virtual buttons through hand gestures. Such gestures are first captured by the HoloLens SDK and then processed by the Service Module of the AR/MR-IoT framework, which is the responsible for sending the switch on/off commands to the smart power outlet and for collecting from it the power consumption data.All the communications that come from the AR/MR application make use of HTTP, which is natively supported by the HoloLens framework. This makes it possible to implement all the communications using the standard APIs provided by Microsoft without any further modifications. This fact, together with the use of MQTT for IoT device communications, allows the framework to provide wide compatibility, which would not be possible without the intervention of the bridge layer to fuse both protocols.In order to describe the inner workings of the developed AR/MR-IoT smart socket system, the next subsections detail with the help of UML sequence diagrams several examples of relevant use cases.To obtain the information on the energy cost per hour, in the case of Spain, it is necessary to access the API of Red Eléctrica Española (REE) . UnfortuON/OFF status.Instantaneous consumption.Instantaneous current.Power factor.Total energy consumed today.Total energy consumed yesterday.Total energy consumed since the socket was installed.The time since the last reset of the smart socket.This use case illustrates the process that is performed when the IoT smart socket notifies its status, which includes the following fields:The sequence diagram in The framework requirements indicated in Due to the need for achieving low-response times and providing that latency is a quantitative requirement that can be evaluated empirically, three sets of tests were performed to measure the response latency of the proposed AR/MR-IoT framework. The first set was aimed at estimating how fast the framework manages interaction requests. The second set of tests quantified the speed of the framework when updating the collected IoT data. Finally, the third set of tests analyzed the main interaction latencies for the use case detailed in As it can predicted, as the number of users and/or devices in the system increases, the load on the framework server increases accordingly, which impacts QoE. However, it is important to note that the load of each individual device remains constant regardless of the number of connected devices. This fact makes it possible to test the system emulating a large number of real devices, which would be really expensive (a pair of Microsoft HoloLens currently costs $ 3500).To validate the use of emulated devices, 30 time-spaced requests were made to the framework first from a real device (a pair of Microsoft HoloLens glasses) and then from an emulated device while the framework server was idle. The obtained results are shown in After validating the use of emulated devices, the rest of test environment was deployed. The cloud server used a virtual machine that run on a DELL Power Edge R415. Such a virtual machine had 4 GB of RAM and a 2-core AMD Opteron processor at 3.1 GHz. In addition, for both sets of tests, up to 500 devices were emulated in a desktop computer  with the objective of estimating the performance limits of the framework without depending on the reduced computational capacity of the embedded hardware devices .It is important to note that the aim of the tests was to assess empirically the framework performance limit for the above-mentioned hardware. Such an evaluation helped to analyze the behavior of the framework with increasing computational loads and made it possible to estimate the hardware that would be necessary for a real environment. To increase the capacity of system, it is possible to use more powerful hardware or scale the system horizontally. To do so, IoT devices can be divided into areas that are managed by cloudlets, which are closer than a cloud, so they manage less IoT devices and provide faster response times. However, such an architecture will require making use of specific protocols to coordinate the different cloudlets and thus sharing the information among them when it is required.In this first set of tests several requests  were sent to the smart power outlet through the REST API. Such requests are internally handled by the AR/MR-IoT framework by first sending them to Node-RED and then to the MQTT broker (Mosquitto). Thus, performance was measured from the moment the request was issued by the HoloLens to when it was parsed and executed by the smart socket.The ’Interaction’ line in This behavior can be further analyzed by observing the ’Interaction’ line in Therefore, the previous experiments indicate that there is a throughput bottleneck on the framework: either Node-RED or Mosquitto slow down interaction request processing. To determine which of both is the responsible, it was measured Mosquitto throughput. The results are depicted in The system throughput does not increase over 200 concurrent clients. This fact suggests that the system is not capable of supporting more concurrent requests. However, for more than 200 concurrent clients what it is relevant is the amount of requests per second that the framework is able to handle, since it establishes the limit of requests that different users can make per unit of time.It is relevant to indicate that tests were finished at 500 test users because the throughput limit was already exceeded. Nonetheless, note that test clients emulate the requests made by a real client, but not the time intervals between each request . Therefore, the actual number of clients supported by framework will depend on the frequency with which such clients make requests.In a second set of tests, data collected from the sensors were requested using the REST API. It is important to note that in these experiments only the AR/MR-IoT framework REST API performance was measured when accessing the IoT data stored on the local database or in memory, so the MQTT broker was not involved.Thorough tests were performed for a different number of concurrent clients. The ’Data request’ line in Finally, the ’Empty request’ line in interaction: it is the total interaction time required to send a command to the smart power outlet.tHTTP: it is the time required by the HoloLens glasses to perform an HTTP request that asks the IoT device to execute the sent command.tMQTT: it is the time required to send the command by using the MQTT protocol.tbridge and tsmart_power_outlet are the times required by the bridge service and the smart power outlet, respectively, to process the command requests.tThere are two different types of response times relevant for the AR/MR-IoT application described in It is important to note that Equation  measuresinteraction for the proposed AR/MR-IoT application under normal operation conditions , it was determined that the average latency when interacting with the smart power outlet was 96.16 ms, with a standard deviation of 17.24 ms.After measuring tThe second type of response time that was measured on the proposed AR/MR-IoT application was related to data requests. For example, such requests can ask for the instant power consumption of a smart power outlet. In this type of requests, the bridge service cache comes into play by serving the information stored in its memory and by requesting periodically updated data from the IoT devices. This reduces considerably the latency as the system does not have to wait for the IoT device to answer the request since it is already cached.data_update stands for the total data update latency, tHTTP is the time related to the necessary HTTP exchange (it includes both the HTTP request and its response) and tbridge is the processing time required by the bridge service.The response times related to data update requests follow Equation , where tdata_update with the proposed AR/MR-IoT application under usual loads , was estimated in 95.64 ms, with a standard deviation of 38.85 ms.The average value obtained for tAs a conclusion, it can be stated that the obtained results show that the proposed AR/MR-IoT framework is able to achieve latencies lower than 100 ms when the server is not under a heavy load. This is enough to fulfill the requirements defined in Finally, it is worth mentioning that the proposed practical system was evaluated when considering a scenario where the server was hosted on a remote cloud. In case of needing faster response times, they can be further decreased by replacing the remote cloud by smaller cloudlets closer to the end devices.This article presented an AR/MR-IoT framework that eases the integration of AR/MR and IoT devices. The framework makes use of well known open-source protocols and tools that together allow for communicating AR/MR and IoT devices dynamically and in real time. After reviewing the state of the art, this paper described the design and implementation of the framework, and provided thorough details on its use for a practical application: an AR/MR-controlled energy monitoring application based on a smart power outlet and Microsoft HoloLens. In order to evaluate the performance on the AR/MR-IoT framework, it was evaluated under different amounts of computational load. The obtained results show that, under normal operation conditions, the framework is able to respond in less than 100 ms to interaction and data update requests. However, for more than 50 requests per second, the operations performed by Node-RED suppose a latency bottleneck whose enhancement requires to manage data access faster.


Based on previously published reference values, women—in contrast to men—frequently had a normal LV size despite severe AR . In a Cox-regression model adjusted for age, LVEDV/BSA and RegF, women were at significantly higher risk for the composite endpoint when compared to men (adj. HR 1.81 (95%CI 1.09–3.03), p = 0.022). Conclusion: In patients with chronic AR, LV remodeling is a hallmark feature in men but not in women. Severity of AR may be underdiagnosed in female patients in the absence of LV dilatation. Future studies need to address the dismal prognosis in female patients with chronic AR.Background: Left ventricular (LV) dilatation is a key compensatory feature in patients with chronic aortic regurgitation (AR). However, sex-differences in LV remodeling and outcomes in chronic AR have been poorly investigated so far. Methods: We performed cardiovascular magnetic resonance imaging (CMR) including phase-contrast velocity-encoded imaging for the measurement of regurgitant fraction (RegF) at the sinotubular junction, in consecutive patients with at least mild AR on echocardiography. We assessed LV size  and investigated sex differences between LV remodeling and increasing degrees of AR severity. Cox-regression models were used to test differences in outcomes between men and women using a composite of heart failure hospitalization, unscheduled AR intervention, and cardiovascular death. Results: 270 consecutive patients  were included. On CMR, mean RegF was 18.1 ± 17.9% and a total of 65 (24.1%) had a RegF ≥ 30%. LVEDV/BSA was markedly closer related with AR severity (RegF) in men compared to women. Each 1-SD increase in LVEDV/BSA (mL/m Left ventricular (LV) dilatation is a hallmark feature of patients with chronic aortic regurgitation (AR) . TransthCardiovascular magnetic resonance imaging (CMR) allows further AR quantification by assessment of regurgitant volume (RegV) and regurgitant fraction (RegF) using phase-contrast velocity-encoded imaging . In addiPrevious studies have linked LV dilatation with incident worsening of clinical status , and theIn this study of consecutive patients with chronic AR undergoing a comprehensive diagnostic workup, including echocardiography and CMR with phase-contrast analysis, we investigated sex differences in (A) the relationship of LV remodeling and AR severity; (B) the prevalence of LV dilatation, based on previously published upper limits of normal; and (C) incident heart failure hospitalization, unscheduled AR intervention, and cardiovascular death. Patients were recruited at a single tertiary referral center with a high-volume multimodality-imaging facility. Consecutive patients with chronic AR were invited to undergo CMR within four weeks of TTE. Both patients with and without a class I indication for aortic valve surgery were eligible for study entry. Preliminary data from this cohort on the diagnostic utility of holo-diastolic retrograde flow in the descending aorta have been published previously . We exclBoard certified cardiologists with extensive expertise in echocardiography performed AR assessment, using high-end scanners, such as GE Vivid S70 . We used an integrated approach in line with current recommendations ,3. In paAll CMR studies were performed on a dedicated 1.5-T cardiac system  using standard protocols . Phase-cPatients were prospectively followed and contacted at a 6-month interval, including on-site visits or telephone contact, respectively. In addition, the nationwide hospital chart system was reviewed to identify the composite of (A) heart failure hospitalization; (B) unscheduled AR intervention in case of clinical worsening equivalent to symptoms of cardiac decompensation; and (C) cardiovascular death. Two study members (AK and JM) served as adjudication committee for each event.2 LVEDV/BSA for men, and 52.2 mm LVEDD, 106 mL LVEDV, and 61 mL/m2 LVEDV/BSA for women [2 LVEDV/BSA for men < 60 years, and 190 mL LVEDV and 94 mL/m2 LVEDV/BSA for men ≥ 60 years. Corresponding thresholds for women were 174 mL LVEDV and 95 mL/m2 LVEDV/BSA (<60 years) and 162 mL LVEDV and 86 mL/m2 LVEDV/BSA (≥60 years) [Continuous data are expressed as mean ± SD and categorical variables are presented in percent and/or total numbers. Wilcoxon rank-sum tests and chi-square or Fisher’s exact tests were used for comparison between male and female patients, as appropriate. Linear regression models, adjusted for age, were used to test the association between LV size parameters and RegF as a metric variable, and standardized beta coefficients are reported for better comparison. To compare the strength of association between LV size parameters and severe AR, we compared sex-specific odds ratios, adjusted for age, per 1 SD increase for each parameter. Consistent with previous literature, we defined severe AR as RegF ≥ 30%. Sensitivity analyses were performed for each published cut-off value, namely a RegF ≥ 27% and ≥33% ,10,11,12or women . On CMR,0 years) . Cox-regTwo different multivariable Cox regression models were run to investigate the impact of sex on clinical outcomes: (A) adjusting for a prespecified set of covariates , and (B) adjusting for all abovementioned variables with a significant impact on a univariable level. Scheduled interventions of the aortic valve were not considered as an event and censored at the time of intervention.In an exploratory step, receiver operating characteristics (ROC) reporting areas under the curve (AUC) based on the Youden index approach were performed to investigate an optimal cut-off for CMR LVEDV/BSA in men and women using the composite endpoint as outcome.p < 0.05, unless stated otherwise. Statistical analyses were performed using STATA 15.1 .Intraclass correlation coefficients (ICC) report interrater variability. For all statistical tests, the level of significance was set to Out of 301 patients, we excluded six patients due to poor CMR quality and 25 due to the presence of a prosthetic valve in the aortic position. Hence, the final study cohort consisted of 270 patients see . No patip < 0.001). 47 (17.4%) patients had a class I indication for AVR  at the time of echocardiography. Baseline characteristics of patients are summarized in p < 0.001 for all). On echocardiography ,3 83 . We found excellent agreement for RegF and RegV between two readers in 20 randomly chosen patients . 2) was associated with a 9.7% increase in RegF in men versus a 5.9% in women . LVEDV/BSA on CMR was markedly closer related with AR severity (RegF) in men compared to women. Each 1-SD increase in LVEDV/BSA . Comparing the different measures of LV size per 1-SD increase, we observed a similar performance between echocardiographic and CMR parameters , Table 3p < 0.001, and 91.3% versus 64.7%, p < 0.001). Also, echocardiography more often classified male patients as having a dilated LV when using LVEDV or LVEDV/BSA as compared to women, although not statistically significant . LVEDD on echocardiography only identified LV dilatation in 45.8% of male patients with severe AR, and in 23.5% in female patients (p = 0.028). In the 65 patients with severe AR, here defined as RegF ≥ 30% on CMR, the presence of LV dilatation varied on echocardiography from 40% for LVEDD to 84.6% for LVEDV, and in CMR from 78.5% for LVEDV to 84.6% for LVEDV/BSA. p = 0.017). Results of the univariable Cox regression are displayed in During a mean follow-up of 32.4 ± 26.5 months a total of 81 events were recorded. By Kaplan–Meier analysis, female patients were significantly more likely to experience the composite endpoint when compared to men (p = 0.014), but not in men (adj. HR per 1-SD increase 1.29 (95%CI: 0.95–1.75), p = 0.104), although showing a similar trend. Further adjustment for RegF on CMR attenuated the effect of LVEDV/BSA in both sexes (adj. HR in women per 1-SD increase 1.24 (0.91–1.69), p = 0.164 versus 1.24 (0.86–1.77) in men, p = 0.246). In that model (including age and LVEDV/BSA), RegF had a trend towards dismal outcomes in both men and women , p < 0.001). By age adjusted Cox regression models, LVEDV/BSA on CMR was significantly associated with a dismal prognosis and women (adj. HR per 1-SD increase 1.44 (95%CI: 1.08–1.91), p = 0.022). In a separate analysis adjusting for factors associated with a significant impact on outcome on a univariable level (p = 0.031) along with RegF (per 1% increase adj. HR 1.02 (1.00–1.03), p = 0.033). In a multivariable Cox-regression model adjusting for the prespecified set of covariates  female sex remained significantly associated with higher risk for events (adj. HR 1.81 (95%CI 1.09–3.03), le level , female 2 in men and 74 mL/m2 in women to be associated with outcome, however, demonstrating moderate overall performance (AUC 0.60 and 0.64 respectively). In an exploratory step, ROC analysis revealed an optimal cut-off for CMR LVEDV/BSA of 108 mL/mAll results were consistent when excluding patients with BAV.In the present study we investigated sex-specific differences in LV dilatation and outcomes in patients with chronic AR. We report three main findings: (1) LV remodeling is more pronounced in men when compared to women; (2) in women LV dilatation is frequently absent when applying standard cut-off values; and (3) women with chronic AR face a more dismal prognosis than men.Accurate assessment of AR is challenging. Current guidelines recommend using an integrated echocardiography approach based on AR jet size, VC, PISA, EROA, and PHT, as the first step ,3. In caAlthough LV dilatation is highlighted as a key finding in both recommendations, it is the only parameter in the integrated algorithm without specific cut-off values ,3. SeverIn contrast to lacking thresholds for the definition of LV dilatation in severe AR, specific cut-off values of LV size are implemented as gatekeeper for indication for surgery in both European and American guidelines. Interestingly, recommendations are based on LVEDD and LV end-systolic diameters, not volumes, and sex-specific thresholds are lacking ,21. In our study, we could demonstrate a clear and linear relationship between increasing RegF on CMR and LV size in men whereas a substantial overlap of LV size is found in women across different degrees of AR severity. Limited data on why LV dilatation is less pronounced in severe AR in women compared to men. In an animal model using induced chronic volume overload, female—in contrast to male—rats did not develop LV dilatation, which is similar to our findings . HoweverAlso, it is important to note that similar to patients with severe aortic stenosis, in whom left ventricular hypertrophy is not a mandatory finding, patients with severe AR may lack LV dilatation and an integrative approach according to current guidelines should be applied. Given the increased prevalence of bicuspid aortic valve and resultant severe AR in males, women have long been under-represented in studies investigating chronic AR ,27,28,292 indexed to BSA [Previous studies have linked LV dilatation with incident worsening of clinical status and established it as an indication for surgery in chronic AR ,12. Currd to BSA ) as gated to BSA ,21. Howed to BSA .2 or higher to be most closely related to the outcome. However, 81% of their study cohort was male, and presence of LV dilatation as well as sex-specific thresholds were not reported [In a CMR study including 113 patients with moderate and severe AR, Myerson et al. reported thresholds for LV size to predict progression of symptoms and incident indication for surgery. They report an LVEDV of 246 mL or higher and a LVEDV/BSA of 129 mL/mA major factor limiting comparison of studies is the lack of data on chronicity of AR. It is known that women present at a later stage with presumably more long-standing disease as compared to men . This maOf note, we could not define an ideal cut-off value for LV size to predict dismal outcome as reflected by modest AUC results in our cohort. These findings highlight that LV size should not be the single gatekeeper for clinical decision making.Several limitations merit comment. Due to the single-center nature, a center-specific bias cannot be excluded. However, advantages of single center studies include the inclusion of a homogenous patient population, adherence to a constant clinical routine, and standardized CMR and TTE examinations. Also, results of our predominantly Caucasian study population may not address the specific diagnostic needs of other ethnicities. Detailed data on AR chronicity was not available but no patient had acute AR or active endocarditis. In addition, sex-specific differences in the clinical management and delayed diagnosis of valvular heart disease in women prior to the study inclusion are likely and should be taken into account as potential confounder. However, this would not explain the absence of LV dilatation in female patients with severe AR, as observed in our study. Furthermore, our study was not designed to test optimal timing of valve intervention in patients with chronic AR. In summary, there are important sex differences in LV remodeling in chronic AR. Increasing AR severity is strongly associated with increase in LV size in men but less pronounced in women. In women, severe AR may be undiagnosed in the absence of LV dilatation based on established thresholds. Future studies are warranted to address the dismal outcome in female patients with chronic AR.


The acoustic reflex (AR) shows promise as an objective test for the presence of cochlear synaptopathy in rodents. The AR has also been shown to be reduced in humans with tinnitus compared to those without. The aim of the present study was twofold: (a) to determine if AR strength (quantified as both threshold and growth) varied with lifetime noise exposure, and thus provided an estimate of the degree of synaptopathy and (b) to identify which factors should be considered when using the AR as a quantitative measure rather than just present/absent responses. AR thresholds and growth functions were measured using ipsilateral and contralateral, broadband and tonal elicitors in adults with normal hearing and varying levels of lifetime noise exposure. Only the clinical standard 226 Hz probe tone was used. AR threshold and growth were not related to lifetime noise exposure, suggesting that routine clinical AR measures are not a sensitive measure when investigating the effects of noise exposure in audiometrically normal listeners. Our secondary, exploratory analyses revealed that AR threshold and growth were significantly related to middle-ear compliance. Listeners with higher middle-ear compliance  showed lower AR thresholds and steeper AR growth functions. Furthermore, there was a difference in middle-ear compliance between the sexes, with males showing higher middle-ear compliance values than females. Therefore, it may be necessary to factor middle-ear compliance values into any analysis that uses the AR as an estimate of auditory function. Cochlear synaptopathy is part of presbycusis, the natural aging process of hearing , as wellMethodological differences between the The use of a broadband probe by some studies . The preregistration stated that a curve would be fitted to the observed data points to allow accurate estimation of the AR threshold and growth. The type of curve was not specified, which, in retrospect, was an oversight. For simplicity, a linear function was instead used to model the data.Forty-eight young adults  were recruited into the study. This number was sufficient to detect an effect size of 1.3 . Two participants had hypercompliant eardrums (compliance >1.5 cm3); however, they were still included in the analyses as their tympanograms displayed a “Type A” shape, and there was no indication of middle-ear dysfunction. The final cohort had a mean age of 27 years (SD = 6 years) with 27 females. All participants gave informed written consent, and all testing materials and procedures were approved by a University of Manchester Divisional Research Ethics Committee (#4768).Of the 45 participants whose data were used in the final cohort, 43 had normal tympanometric results  had present distortion product otoacoustic emissions, defined as having a response amplitude >0 dB SPL at 3 or more test frequencies. Distortion product otoacoustic emissions were obtained to screen for gross outer hair cell dysfunction, not currently detectable using pure-tone audiometry, which could account for reduced ARs.ARs were measured from the right ear using a GSI Tympstar diagnostic middle-ear analyzer with Grason KR-Series clinical ear tips, as this was the same ear from which audiometric thresholds were evaluated. Calibration was performed before each test session using a 2-cc coupler. Tympanograms and ARs were measured using a 226 Hz probe tone (trains of ∼40 ms pulses). Three 1.5 s elicitors—0.5 kHz pure tone, 2 kHz pure tone, and 0.4–4 kHz BBN—were presented ipsilaterally and contralaterally, creating a total of six stimulus conditions. The six conditions were tested in a pseudorandom order with each condition tested three times. The initial presentation level was 70 dB HL for tonal stimuli and 60 dB HL for the broadband stimulus. In each “run,” a reflex threshold response was first determined and then three further measurements were made at 5, 10, and 15 dB above the initial estimated threshold. The reflex threshold was defined as the lowest stimulus intensity resulting in a reduction in middle-ear compliance of ≥0.02 ml, with appropriate morphology and no evidence of significant measurement artefact. The process of measuring the threshold and the three suprathreshold reflexes was then repeated twice more before moving on to the next condition. If a significant measurement artefact was observed during a response period, the presentation was repeated.3/5-dB). We chose to express the change relative to a 5-dB stimulus level increase as we felt that this could be a more intuitive way to think of the growth change for clinicians.A linear regression was performed on each of the three runs for a condition, and these functions were then averaged. Using this estimate of the AR, the stimulus level (in dB HL) at which tympanometric compliance would change by 0.02 ml was extrapolated, and this was taken as the AR threshold. The slope of the function was used to calculate the growth of the AR, and this is represented as the rate of change in compliance for each 5 dB in stimulus level . Although there is a reasonable spread of noise exposure histories, there is only one person with a high exposure (>2) and only a small number with values below 0.5.ear had two levels  and the factor elicitor had three levels . Tests of sphericity revealed no violation of assumptions and a main effect of both ear, F = 37.41, p < .001, and elicitor, F=4.454, p = .016, with no significant interaction, F = 2.99, p = .059. Simple main effects analysis confirmed that there was a significant difference between the ear of elicitor presentation for BBN and 0.5 kHz elicitors.A two-way repeated-measures analysis of variance was performed on measures of growth, in which the factor p < .001 and p = .003 for BBN and 0.5 kHz elicitors, respectively). There was no difference in AR growth measured ipsilaterally or contralaterally for the 2 kHz elicitor. For contralateral presentations, AR growth was significantly greater for 2 kHz and 0.5 kHz elicitors relative to a BBN . There was no difference in contralateral reflex growth for the two tonal elicitors (p = .28). For ipsilateral presentations, there was no difference between reflex growth for any of the elicitors .Ipsilateral estimates of AR growth were larger than contralateral growth , the cohort was treated as a single group, with noise exposure as a continuous variable. The main research question concerned whether AR threshold or growth varied significantly as a function of estimated lifetime noise exposure. Although this cohort had absolute hearing thresholds ≤20 dB HL at each audiometric frequency, it is still possible that audiometric sensitivity is related to the strength of the response. To investigate this, correlation coefficients were calculated between reflex thresholds and the most relevant audiometric test frequency which were 0.5 kHz and 2 kHz for the 0.5 Hz and 2-kHz tonal elicitors, respectively. For the BBN conditions, the reflex thresholds were correlated with 1 kHz audiometric thresholds (as this is the frequency of the noise most effectively transmitted by the transducers used), the pure-tone average calculated over the range 0.25–8 kHz, and also thresholds at 16 kHz. To keep the number of comparisons down, only thresholds were considered, and contralateral conditions were used as they showed better overall response rates than the ipsilateral conditions. The correlations are shown in r = –.24, p = .12). Furthermore, as a priori methodological decision of whether threshold or growth will be used to quantify the reflex and whether reflexes are measured ipsilaterally or contralaterally are critical in determining how an individual’s reflex will be quantified relative to the rest of the cohort.AR thresholds and growth, elicited by a contralateral BBN, showed no significant relation to each other  and AR growth  for a contralateral BBN elicitor. Therefore, the more compliant a listener’s eardrum, the lower their threshold and the steeper their growth  and that with threshold strengthens (p = .007).Middle-ear compliance (the peak of the recorded tympanogram) was significantly correlated with both AR threshold  and female listeners . However, for AR growth, males showed steeper functions than females . Given the relation between compliance and AR suggested by the previous analysis, a test was also run to determine whether there was a difference in middle-ear compliance between the sexes: Welch’s t = 2.56, p = .017, indicating males show significantly higher levels of compliance.There was no a priori reason to expect differential AR responses between male and female listeners. However, in the cochlear synaptopathy literature, there have been questions raised over whether males and females are similarly vulnerable, given differences in their high-frequency hearing  and the rs = .60, p < .001). There were no significant relations between age and the contralateral BBN reflex threshold , reflex growth  or between age and peak tympanometric compliance .The existence of noise-induced cochlear synaptopathy remains a contentious and unresolved issue. However, there is growing evidence that cochlear synapses are lost across the cochlea length as the system ages , estimates of threshold and growth were not reliably correlated with each other and so the choice of how to quantify the response will also lead to different estimates of underlying synaptopathy in a listener, if the AR is being used for this purpose.Whilst this study had a range of participants with varying degrees of noise exposure, our numbers of highly exposed listeners, such as people who very frequently attend loud music events (NESI >2), were limited. As noted in the Introduction section, there are different approaches to measuring the reflex which use different probe stimuli. Whilst the conclusions drawn from the current study may be specific to the 226 Hz probe tone used, it is important for future studies to consider these issues and the extent to which they may reduce the accuracy of any measurements. Aside from the specific probe used, there are other technical considerations which may be important. For example, wideband systems typically use in-situ forward-pressure calibration to allow greater accuracy over what sound levels actually reach the tympanic membrane. Some clinical systems also offer the ability to correct presentation levels based on estimates of individual ear canal volume and physiology; however, the system we used did not. This lack of subject-specific calibration leads to increased between-subject variability which in turn will affect the power of the study. We conclude here that the clinical protocol, using a 226 Hz probe tone, is not suitable for quantitative measures of the reflex in healthy listeners, under the assumption that a subset of the cohort has sustained subclinical audiological changes. However, it remains possible that if some of these technical aspects were amended, the 226 Hz probe tone could provide sufficient statistical power across a group of listeners to quantify differences in the response related to underlying physiological changes. It may be that different tones  may be a better choice of probe or that different eliciting stimuli may produce less variability which is not from the source of interest . However, it remains our view at the moment that any measures should look at the consistency of the response across different recording montages, assess if the reflex needs to be normalized relative to tympanometric peak compliance and if there are differences as a function of sex.The underlying assumption behind the work described was that if the AR is a reliable marker of cochlear synaptopathy, then, even if the wideband measure is more sensitive to detecting this, given sufficient statistical power, the clinical approach would be able to detect this effect. There are numerous examples in the literature which highlight the idiosyncratic spectral shifts which make up the reflex when using a wideband probe; for example, some listeners can have minimal reflex magnitude at 226 Hz but a very large reflex elsewhere in the spectrum. Examples such as the shifts reported in There are currently little available data directly comparing the two AR measures. If it were subsequently established that the two probe choices are not equivalent, then the assumptions underpinning the current study would be violated and further caution would be required when interpreting the data.The work by The measures of AR threshold and growth were found not to be correlated with each other, and therefore, it may well be of importance which measure—probe stimulus, and/or side of stimulation, is used as a dependent variable in future studies. Also, in this group of healthy listeners, a BBN elicitor produced the largest response (lowest threshold), and contralateral BBN was the only elicitor to yield a 100% response rate. However, the correlation between the ipsilateral and contralateral responses was weakest for the BBN (irrespective of whether threshold or growth is considered). Therefore, not only is it important to consider whether a measure of growth or threshold is used to quantify the response but also whether the response is measured using an ipsilateral or contralateral elicitor. The data presented in this study suggest that listeners with the strongest response in one ear might not have the strongest response using a different stimulus ear or frequency, which is a concern as this methodological choice will therefore impact the pattern of results seen.Furthermore, middle-ear compliance was found to be a significant predictor of subsequent reflex strength for all elicitors and both measurement montages. In the current study, males showed larger levels of middle-ear compliance, and this was related to lower thresholds and steeper growth functions. The relation between sex, middle-ear compliance, AR threshold, and growth is unclear. The standard, clinical methodology for obtaining the AR is not appropriate for quantitative measures of the response.The reflex measured may be related to the compliance of the middle ear.Growth and threshold estimates of the reflex are not related to each other.Ipsilateral and contralateral estimates of the response evoked with the same elicitor are not strongly predictive of each other.There is no evidence that the AR is related to the degree of estimated lifetime noise exposure when using a 226 Hz probe. There are a number of clear experimental designs and methodological considerations which should be factored into any future studies which seek to use the AR as a quantitative research tool which are as follows:


Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented reality (3D-AR) is an emerging technology that has the potential to assist physicians with spatial reasoning during clinical interventions. The most intriguing applications of 3D-AR involve visualizations of anatomy or surgical plans that appear directly on the patient. However, commercially available 3D-AR devices have spatial localization errors that are too large for many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients, including manual registration, computer vision-based registration, and registrations that incorporate external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of approximately 2 mm are feasible. 3D-AR device limitations due to the vergence-accommodation conflict or other factors attributable to the headset hardware add on the order of 1.5 mm of error compared to conventional guidance. Continued improvements to 3D-AR hardware will decrease these sources of error. I.Spatial reasoning is one of the primary challenges of clinical interventions across medical disciplines. Many tools and workflows exist to help clinicians visualize and understand the relationship between critical anatomy and surgical tools. Image-guided procedures incorporate pre-operative imaging or real-time imaging to help physicians guide surgical tools to treatment targets and avoid damaging adjacent structures and tissue. These tools are utilized for intracardiac procedures, neurosurgery, biopsies, and many other clinical procedures. A key challenge of image-guided procedures is that information is typically displayed on a two-dimensional (2D) screen, requiring clinicians to mentally relate the images to the three-dimensional (3D) patient and to frequently switch their attention between the patient and display Augmented reality (AR) is an emerging technology that has the potential to help physicians with spatial reasoning during clinical interventions While the landscape of 3D-AR hardware is increasingly diverse, the Microsoft HoloLens  was one of the first commercially available 3D-AR devices. Notable hardware features of the HoloLens and other 3D-AR devices are presented in Studies of 3D-AR devices have demonstrated their utility as clinical visualization tools. One advantage of 3D-AR displays is that they provide the user with a high degree of flexibility for positioning and scaling views of clinical data. A prospective study evaluated the use of the HoloLens as an alternative to conventional monitors for endoscopic ureteroscopy in immersive simulated procedures. The evaluation of 72 participants found that procedural times and Objective Structured Assessment of Technical Skill  scores improved with the HoloLens compared to conventional monitors. 95% of participants agreed that the HoloLens is feasible to introduce clinically and will have a role within surgery While these results are promising for the role of 3D-AR devices as clinical visualization tools, there are many intriguing applications of 3D-AR that require displaying or overlaying important information at spatially precise locations. In a clinical context, this can mean visualizations of anatomy or surgical plans that appear directly on the patient. The accuracy of these visualizations is a key factor in determining whether they can be used to guide clinical interventions. Commercially available 3D-AR devices were designed to ensure that holograms appear spatially stable as users move around in room-sized environments. Localization errors on the order of a few centimeters are common when using these devices. While these spatial errors are acceptable for many basic AR experiences, errors of this size are incompatible with many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. A commercial example includes the Augmedics xVision system, which uses a custom 3D-AR headset The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients including manual registrations, computer vision-based registrations, and registrations that incorporate external tracking systems. Many of the approaches cited in this review have been developed in other fields such as robotics At the time of writing, several recent advances in AR-related hardware have yet to make an appearance in clinical settings but are sure to move clinical applications of 3D-AR forward. 3D-AR visual fidelity will benefit from improvements in display hardware , and tracking accuracy will be improved by advances in sensor hardware . The HoloLens 2 was recently released and will likely make an impact on clinical AR, as evidenced by a concept collaboration between Microsoft and Philips II.Panel 1). This approach was evaluated in a surgical environment by positioning a holographic view of the patient’s scapula over the patient’s shoulder during a successful reverse shoulder arthroplasty procedure The simplest approach to registering 3D-AR holograms to a patient is to manually perform the registration. The general principle is that the user sees a hologram in the headset and manually adjusts the hologram position using software controls until the hologram is spatially aligned with the physical structure it represents (Panel 2). While manual registration is a simple and relatively imprecise technique, this approach has already been applied clinically in a study of bedside external ventricular drain insertion Panel 3).While manual registrations are often performed by positioning holograms over physical targets, the same result can be achieved by moving physical objects to holographic targets. A major challenge of this approach is that judging the relative depths of physical and virtual objects is very difficult. When touching physical objects, the user feels tactile feedback when an object contacts its target. Additionally, a physical object will occlude the view of another physical object behind it. This visual occlusion provides an important depth cue. Holograms have no tactile feedback, and current-generation hardware lacks the spatial mapping resolution to integrate occlusion of small objects. An interesting solution to this problem was to align holographic cubes with handheld cubes that were tracked by an external camera III.While manual registrations have proven useful, the registration process adds time to procedures and introduces the possibility of human error impacting registration accuracy. One of the primary tools available to help automate registrations is the front-facing camera on the HoloLens. This camera can record photos and videos, but it can also be utilized by computer-vision algorithms. The general principle of computer vision tracking approaches is that a 2D image or 3D object with known dimensions is tracked by a digital camera, and a software algorithm determines the position and orientation of the tracking target relative to the camera. Because the front-facing camera on the HoloLens is at a fixed location relative to the display, coordinates determined using the camera are directly related to display coordinates. The HoloLens software development kit (SDK) allows developers to access the camera’s spatial position and perspective The computer vision capabilities of 3D-AR devices can be leveraged for registrations by placing computer vision targets at known locations relative to clinical workspaces or patient anatomy. This approach is very flexible and has been used in a wide variety of applications. One application of this approach utilized a patient-specific 3D-printed registration guide A computer vision target was attached to an endoscope as part of an Augmented Reality Assistance for Minimally Invasive Surgery (ARAMIS) system IV.Image registration targets are also useful for mitigating the effect of spatial drift in 3D-AR applications. Drift occurs because of limitations in the accuracy of 3D-AR headset spatial mapping systems. An early study on the suitability of 3D-AR hardware for clinical applications quantified spatial drift The effect of computer vision tracking on hologram stability was evaluated in a study using a 3D-printed skull phantom V.In many cases, registration of hologram to patient anatomy requires precise 3D localization of anatomical landmarks or externally attached fiducial markers. Digitizing 3D points using the HoloLens can be achieved without any external hardware by taking advantage of the SLAM functionality. The SLAM functionality creates spatial maps of the environment, and the user can record 3D coordinates by pointing the visual gaze cursor at a location and performing a voice command or hand gesture to record the intersection of the visual gaze with the surface the user is looking at. However, the spatial maps of the environment generated by current-generation 3D-AR devices are optimized for flat surfaces such as walls, ceilings, and tables, and they cannot represent patients or clinical tools with the accuracy required for most clinical applications. Therefore, additional measures must be taken to stabilize holograms and register images to patients. One solution used to digitize 3D coordinates with the HoloLens headset was to affix an image tracking target to a digitizing pointer of known geometry VI.While the built-in SLAM and computer vision capabilities of 3D-AR devices are useful for many clinical applications, external tracking hardware is often required to improve the precision of spatial tracking and localization. For example, one study evaluated 3D-AR for guiding a flexible needle to a 2 cm phantom target Two common systems used for high precision tracking and localization are optical and electromagnetic (EM) tracking systems M, are related to the coordinates of the headset view origin, H, by a rigid transformation with rotation R and translation t: One way to incorporate external tracking systems into 3D-AR applications is to affix hardware to the headset to simultaneously track the headset and surgical tools or other objects that will be visualized in the headset. The primary challenge to this approach is that accurate hologram visualization depends on the position and orientation of the headset view origin, which cannot be precisely determined by physically examining the headset. When tracking hardware is attached to the headset, the coordinates of the tracking markers, Some studies utilized the HoloLens spatial tracking to solve for the transformation from headset trackers to view origin. An application of 3D-AR to remotely controlled spray painting used a four-second calibration where the HoloLens was moved in 3D space while tracked by an optical camera system While optical tracking systems track large volumes well, registrations involving EM tracking systems tend to favor approaches that involve touching holographic points with a tracked pointer An alternative approach to solve for the transformation in VII.In some cases, anatomical landmarks or registration fiducials are not available for registering pre-operative imaging to a patient. One solution to this problem that has been explored is the use of depth cameras. Depth cameras can be used for mapping and registrations by extracting sets of 3D points from imaged surfaces. Depth maps of head phantoms have been registered with surfaces generated from CT data VIII.A key question concerning the use of 3D-AR guidance for clinical procedures is: what are the accuracy limits of these registrations? Unfortunately, because holograms lack tactile feedback and visual occlusion depth cues, quantifying the accuracy of these visualizations is not straightforward. A variety of different approaches for measuring registration accuracy have been explored. The “mixed reality capture” feature of the HoloLens has been used for quantifying registration accuracy Other registration accuracy assessments generally require human perception. These evaluations often measure the accuracy of the user when performing a clinically relevant task with 3D-AR guidance. While these assessments do not purely measure the quality of the AR visualization, what they do capture—the ability of users to perform clinically relevant tasks with AR guidance—is ultimately the most important metric. One common method for assessing registration accuracy is to use a tracked pointer to touch holographic targets and measure the distance error from the intended target . The VAC is caused by the optical distance of the displays in devices such as the HoloLens. When users look at virtual objects displayed in the HoloLens headset, their eyes must accommodate to a focal distance of 2 m XI.Another visual concern for some 3D-AR applications is the “psychophysical issue.” This issue arises when a properly registered hologram that is spatially behind or beneath a physical object appears to the user as being in front of the physical object XII.Clinical applications of AR will benefit from improvements to 3D-AR hardware. Compared to the first-generation HoloLens, the HoloLens 2 features a higher-resolution color video camera and higher-resolution time-of-flight depth camera. Additional improvements in hardware will likely emerge from the smartphone supply chain. Smartphones are the most widely available AR devices, and smartphone hardware has driven innovation in inertial measurement units (IMUs) and camera optics. The latest iPhones now feature light detection and ranging (LiDAR) hardware for depth-sensing. These tracking hardware improvements can directly impact 3D-AR applications that directly rely on 3D-AR device tracking, such as manual registrations. Improvements to the computational power of 3D-AR devices are also likely to move the field forward. Many aspects of registrations, including computer vision algorithms, are computationally intensive. The first-generation HoloLens had a 4-core 1 GHz CPU, while the HoloLens 2 has an 8-core 2.6 GHz CPU. The additional computational power combined with improved camera quality may enable an expanded role for computer vision tracking markers. This could include the use of additional markers, more stable tracking when the markers are farther from the device, or better performance when the markers are moving relative to the 3D-AR device. Display hardware is also rapidly advancing. The HoloLens 2 displays have a resolution of XIII.A growing body of literature supports the use of 3D-AR devices for guiding clinical interventions. The intuitive visualizations of 3D-AR devices have the potential to make difficult, skill-intensive procedures much more approachable. A study of 3D-AR guidance for CT-guided lesion targeted found that AR guidance elevated the performance of all users and helped novices perform as well as experienced clinicians A key challenge to using these devices in a clinical context is performing an accurate registration to the clinical workspace. One of the most accurate ways to register the 3D-AR devices to external coordinate systems is to use image targets and computer vision algorithms to determine the position of the target relative to the headset. This approach can correct for drift in the headset’s spatial mapping and can also be used to perform accurate registrations with external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of around 2 mm are feasible.Error in 3D-AR device registration is due to contributions from sensor accuracy and resolution, manufacturing tolerances, human visual anatomy, and psychophysical effects. Improvements in hardware design will continue to make incremental improvements to the maximum attainable accuracy, while careful design can continue to mitigate human factors.Competing InterestsThe content is solely the responsibility of the authors and does not necessarily represent the official view of the NIH. Christopher M. Andrews, Alexander B. Henry, Ignacio M. Soriano, and Michael K. Southworth are employees of and hold equity in SentiAR. Jonathan R. Silva serves on the Board of Directors, a Consultant, and holds equity in SentiAR.


TH expression in the lower brainstem and adrenal glands and β2-AR expression in the lower brainstem were analyzed by western blot analyses. In the leukocytes, TH and β2-AR expression was evaluated by flow cytometry before and after chronic treatment with the centrally-acting sympathoinhibitory drug clonidine. Western blot analyses showed increased TH and β2-AR expression in the lower brainstem and increased TH in adrenal glands from SHR compared to normotensive Wistar Kyoto rats (WKY). Lower brainstem from SHR treated with clonidine presented reduced TH and β2-AR levels, and adrenal glands had decreased TH expression compared to SHR treated with vehicle. Flow cytometry showed that the percentage of leukocytes that express β2-AR is higher in SHR than in WKY. However, the percentage of leukocytes that expressed TH was higher in WKY than in SHR. Moreover, chronic treatment with clonidine normalized the levels of TH and β2-AR in leukocytes from SHR to similar levels of those of WKY. Our study demonstrated that the percentage of leukocytes expressing TH and β2-AR was altered in arterial hypertension and can be modulated by central sympathetic inhibition with clonidine treatment.The sympathetic nervous system (SNS) plays a fundamental role in the pathophysiology of cardiovascular diseases, including primary arterial hypertension. In this study, we aimed to investigate whether the expression of the rate-limiting enzyme in catecholamine synthesis, tyrosine hydroxylase (TH), and the β Cardiovascular diseases are the leading cause of mortality worldwide, representing approximately 31% of all-cause mortality .Central sympathetic hyperactivity and the resulting activation of the sympathetic outflow to the blood vessels, the heart, and the kidneys is a hallmark in hypertensive patients  and in eConsequently, the assessment of the systemic levels of sympathetic activity appears to be crucial in the management of primary hypertension, allowing for the identification of a subset of hypertensive patients presenting sympathetic overactivity. In these patients, the modulation of central sympathetic activity using centrally acting antihypertensive drugs could be especially beneficial. Human regional sympathetic activity can be studied by microneurography of postganglionic sympathetic efferences directed to the skeletal muscle vasculature and by noradrenaline spillover, which is a measurement of organ-specific catecholamine release into plasma, together with direct measurement of plasma catecholamines and spectral analysis of heart rate signals . However2-adrenergic receptor (β2-AR) by catecholamines leads to noradrenaline release (1-AR. α2-Adrenergic receptors (inhibitory auto-receptors) are also present in the noradrenergic nerve endings and are involved in the regulation of noradrenaline release through a negative feedback mechanism mediated by the neurotransmitter itself  is a rate-limiting enzyme in catecholamine synthesis and a key component of the central and peripheral SNS . The act release  and indur itself . During chanisms , resultid access .The interactions between the autonomic nervous system and the immune system are considered to be bidirectional. Primary and secondary lymphoid organs have rich sympathetic innervations that are responsible for the release of noradrenaline and other neurotransmitters . CatechoThe presence of components of the sympathetic system in leukocytes has been related to central sympathetic hyperactivity ,15. Furt2-AR is higher in the mononuclear cells of hypertensive patients than in those of normotensive controls; moreover, there is a significant correlation between β2-AR density and mean arterial blood pressure in primary hypertension .Male Wistar Kyoto rats (WKY)  and spontaneously hypertensive rats (SHR) of the Okamoto-Aoki strain , aged 12 to 14 weeks and weighing 200-250 g, were used in this study. The animals were housed at a maximum of 4 individuals per cage, kept in a specific-pathogen-free (SPF) room at controlled conditions of light (12-h light/dark cycle) and temperature (22±1°C), and were provided sterilized water and standard rat chow -1·day-1 clonidine  dissolved in distilled water once per day for 28 days by gavage. The clonidine dose was chosen according to previous studies from our laboratory, showing that these doses were adequate to inhibit central sympathetic activity and also to normalize arterial pressure in hypertensive animals  and heart rate were measured in conscious animals using a computerized tail-cuff plethysmography system, a noninvasive blood pressure acquisition system . The rats were adapted to the pre-warmed tail-cuff apparatus for three consecutive days to acclimatize them and to avoid stress during the experimental procedures. Each session consisted of 15 cycles of measurements; the first 5 cycles were acclimation cycles and the data were not used, whereas the following cycles were used for the calculation of mean values of hemodynamic parameters.Total blood was collected in microtubes containing EDTA . The quantification of leukocytes was performed by impedance detection using an automated hematological Poch Analyzer 100-iV DIFF .ip), the blood was collected by cardiac puncture, and the adrenal glands and the lower brainstem were removed for western blot analysis.The animals were sacrificed with an anesthetic overdose  according to the manufacturer's protocol. ELISA assay is widely used in experimental studies and accepted in clinical practice. Moreover, it has already been shown that the ELISA assay is accurate, sensitive, specific, and precise in the evaluation of plasma catecholamines  were processed for extracellular β2-AR and intracellular TH analysis. For TH evaluation, permeabilization was performed with 0.1% saponin. Nonspecific binding was blocked with inactivated goat serum at 4°C for 30 min and incubation with primary antibody  or rabbit anti-receptor β2-adrenergic at dilution of 1:50  for 20 min. After washes with phosphate buffered saline (PBS), the incubation with secondary antibody conjugated to Alexa Fluor 488 at 1:1000 dilution  was performed for 20 min. The samples were fixed with 2% PFA, and analysis was carried out in the Cyan ADP Analyzer system . Biological samples were analyzed in a minimum of three technical replicates. The data were analyzed with Summit software version 4.3 (Beckman Coulter).Leukocytes (10g for 20 min at 4°C and the supernatant was kept frozen at -80°C. Proteins (50 µg/lane) were separated by SDS/PAGE (12%) and analyzed by immunoblotting with specific primary antibodies [rabbit anti-tyrosine hydroxylase at 1:5000 dilution (58 kDa)  or rabbit anti-receptor β2-adrenergic at 1:2000 dilution (46 kDa) ]. The immunogen utilized by the antibody manufacturer was a synthetic peptide encompassing a sequence within aa 340-413, which minimizes the cross-reactivity with different subtypes of β-ARs . For loading control, we used mouse anti-glyceraldehyde 3-phosphate dehydrogenase  monoclonal antibody . The membranes were incubated with secondary goat anti-mouse IgG HRP-labeled antibody  or with secondary goat anti-rabbit IgG HRP-labeled antibody  at 1:10000 dilution for 1 h at room temperature, followed by incubation with chemiluminescent kit Supersignal  and exposition to X-ray film. Biological samples were analyzed in a minimum of three technical replicates. The X-ray film bands densitometry analysis was performed with the software Image Studio Lite version 4.0  and standardized by the GAPDH density. Considering that a unique band should be visible in western blot assays, it was technically impossible to determine β2-AR expression in the adrenal glands, since an extra band of 20 kDa was observed in the adrenal gland samples. Therefore, these data were not included in the results. Moreover, in the leukocytes, the small protein quantity obtained from the samples precluded the assessment of TH and β2-AR by western blot analysis with the available antibodies.Proteins were extracted from 100 mg tissue/mL lysis buffer . The samples were sonicated twice and centrifuged at 12,000 t-test, and if data did not present a normal distribution, a two-tailed Mann-Whitney test was used. One-way ANOVA followed by Tukey's post hoc test was employed in the analyses of more than two groups with normal distribution. Kruskal-Wallis test followed by Dunn's post-test was used in the evaluations among three groups with abnormal distributions. The Pearson correlation coefficient was used to measure the strength of the linear relationship. Differences with P values of less than 0.05 were considered significant. All analyses were performed using the statistical program GraphPad Prism 5 .Data are reported as means±SD or median (interquartile range). D'Agostino's K-squared test and Pearson's normality tests were used to evaluate normality of the data. Statistical analyses between two groups with normal distribution were performed using a two-tailed unpaired As expected, hypertensive animals treated with vehicle (SHR+VEH) presented higher systolic blood pressure and heart rate than did normotensive WKY rats  and SHR+VEH (P<0.001). We observed significant reductions in plasma noradrenaline and adrenaline levels after 28 days of clonidine treatment compared with SHR+VEH  than in WKY controls . Compared with non-treated SHR (0.85±0.17), clonidine treatment in SHR induced a significant reduction in TH levels in the adrenal glands  .2-AR (0.68±0.13) were more highly expressed in the SHR+VEH group than in the normotensive control groups . Both protein levels in the brainstem were decreased in the clonidine group  compared with SHR+VEH   and βctively) .3 leukocytes/mm3, SHR+VEH had 8.14±1.95×103 leukocytes/mm3, and SHR+CLO contained 8.72±2.97×103 leukocytes/mm3  and SHR+VEH rats  .We observed that the median fluorescence intensity of TH was higher in the leukocytes of normotensive WKY rats (5.6±2.1) than in the leukocytes of SHR+VEH  .2-AR was observed in WKY rats (6.0±3.9%) than in SHR+VEH . Compared with SHR+VEH , clonidine treatment resulted in a reduction in β2-AR-positive measures in leukocytes (15.0±8.1%). However, clonidine treatment induced a higher percentage of β2-AR-positive cells than was observed in normotensive WKY  (A lower percentage of leukocytes positive for β P<0.05) .2-AR was lower in WKY leukocytes (0.94±0.30) than in SHR+VEH leukocytes  (We found that the median fluorescence intensity of βP<0.001) .2-AR expression in leukocytes with hemodynamic parameters  (The correlations of TH and βrameters  and caterameters  were evarameters  and hearrameters . There w=0.0001) , which w=0.0001) .vs systolic arterial pressure (r=-0.4571 and P=0.0372) and vs heart rate (r=-0.5176 and P=0.0163); percentage of leucocytes positive for β2-AR vs systolic arterial pressure (r=0.6217 and P=0.0412) and heart rate (r=0.4702 and P=0.1230); percentage of leucocytes positive for TH vs plasma noradrenaline concentrations (r=-0.5226 and P=0.0151).There was an inverse linear relationship  between the percentage of leucocytes positive for TH and plasma noradrenaline levels evaluated in normotensive and hypertensive animals after long-term treatment with clonidine or vehicle . There wIt is recognized that sympathetic hyperactivity is a therapeutic target in the management of arterial hypertension , not onlHowever, the assessment of sympathetic activity in patients is performed using methods with low sensitivity or that are difficult to implement in clinical practice , includiPlasma and urine catecholamine measurements are noninvasive and are the most commonly used techniques to evaluate sympathetic activity in clinical practice. However, these methods depend on the levels of secretion, tissue clearance, and re-uptake of catecholamines. For instance, it has already been demonstrated that plasma noradrenaline levels correspond to noradrenaline concentrations released from presynaptic neurons and, consequently, a sympathetic nerve activity marker, only when norepinephrine re-uptake transporter is blocked . MoreoveMicroneurography is the recording of postganglionic muscle sympathetic nerve activity, usually studying peroneal or brachial nerves. These local evaluations are invasive and complex to perform in the clinical setting . Another2-AR in peripheral blood leukocytes and the relationship between the expression of these proteins in the adrenal gland or lower brainstem and systolic blood pressure.Therefore, it is important to search for new markers of systemic sympathetic activity. In this context, we analyzed the expression of TH and βTH appears to be a key component in the development of hypertension in SHR, since higher TH mRNA levels are detected in the adrenal gland and medulla oblongata of SHR than in those of WKY . Moreove2-AR selective antagonist also prevented the increase in arterial pressure induced by C1 region electrical stimulation . In this context, it has already been demonstrated that chronic catecholamine elevation induces a decrease of the number of plasma membrane β2-AR in lymphocytes. This phenomenon was shown to be reversible upon normalization of plasma catecholamine levels in patients with pheochromocytoma (2-AR in the leukocytes.Our results confirmed the presence of hyperactivity of the SNS in arterial hypertension, as demonstrated by increased TH and βmocytoma . Neverth2-AR (2-AR in total leukocytes using flow cytometry, and showed that leukocytes from both WKY and SHR expressed TH and β2-AR. It is important to note that platelets were eliminated from the analysis in the side and forward scatter parameters of the flow cytometry assays by the different sizes of these cells; moreover, circulating endothelial cells are quite rare in peripheral blood, as described previously (Previous studies demonstrated that mononuclear leukocytes synthesize catecholamines and express β2-AR . In the eviously . Thus, b2-AR expression in leukocytes with systolic blood pressure. There was also a higher number of leukocytes positive for TH in normotensive animals than in hypertensive animals. On the other hand, a lower percentage of leukocytes presented β2-AR in normotensive animals; therefore, higher levels of leukocytes positive for the receptor were present in animals with higher systolic blood pressure values.Considering the groups of normotensive and hypertensive animals as a single population, we found an inverse correlation for TH expression and a direct correlation for βThe use of biomarkers with a negative correlation with the biological process has been previously proposed . In this2-AR than those from normotensive controls. In that study, a significant correlation of β2-AR density in mononuclear cells with mean arterial pressure was also found. This analysis was performed by grouping, as a single population, the data of normotensive patients together with that of hypertensive patients (2-AR receptor expression, alterations in α1-adrenergic receptor expression were observed in peripheral blood lymphocytes from patients with essential hypertension and from the animal model of primary hypertension of the SHR (Our results corroborate data from the literature , demonstpatients . In addi the SHR . It was  the SHR .2-AR did not exclude a possible direct effect of clonidine on these cells. To evaluate this effect, in vitro analyses of leukocyte treatment with clonidine should be performed. However, the incubation of peripheral blood mononuclear cells with 4 μM clonidine did not alter the density of β-AR without distinction between β1- and β2-AR (Our results on the expression of leukocytes positive for TH and βnd β2-AR , indicatThe causes and the biological significance of the relative reduction of TH expression in the peripheral blood leukocytes of hypertensive animals, compared to the normotensive counterpart, remain to be elucidated. This could be related to a counter-regulatory mechanism resulting from the increased plasma levels of catecholamines in hypertensive animals.2-AR was altered in arterial hypertension and can be modulated by central sympathetic inhibition with clonidine treatment, thus suggesting that the expression of these proteins could be candidates for the development of peripheral markers of the central sympathetic activity in primary arterial hypertension.In summary, our results demonstrated that the percentage of leukocytes expressing TH and β