Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> Augmented reality systems allow users to interact with real and computer-generated objects by displaying 3D virtual objects registered in a user's natural environment. Applications of this powerful visualization tool include previewing proposed buildings in their natural settings, interacting with complex machinery for purposes of construction or maintenance training, and visualizing in-patient medical data such as ultrasound. In all these applications, computer-generated objects must be visually registered with respect to real-world objects in every image the user sees. If the application does not maintain accurate registration, the computer-generated objects appear to float around in the user's natural environment without having a specific 3D spatial position. Registration error is the observed displacement in the image between the actual and intended positions of virtual objects. > <s> BIB001 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> Enhanced reality visualization is the process of enhancing an image by adding to it information which is not present in the original image. A wide variety of information can be added to an image ranging from hidden lines or surfaces to textual or iconic data about a particular part of the image. Enhanced reality visualization is particularly well suited to neurosurgery. By rendering brain structures which are not visible, at the correct location in an image of a patient''s head, the surgeon is essentially provided with X-ray vision. He can visualize the spatial relationship between brain structures before he performs a craniotomy and during the surgery he can see what''s under the next layer before he cuts through. Given a video image of the patient and a three dimensional model of the patient''s brain the problem enhanced reality visualization faces is to render the model from the correct viewpoint and overlay it on the original image. The relationship between the coordinate frames of the patient, the patient''s internal anatomy scans and the image plane of the camera observing the patient must be established. This problem is closely related to the camera calibration problem. This report presents a new approach to finding this relationship and develops a system for performing enhanced reality visualization in a surgical environment. Immediately prior to surgery a few circular fiducials are placed near the surgical site. An initial registration of video and internal data is performed using a laser scanner. Following this, our method is fully automatic, runs in nearly real-time, is accurate to within a pixel, allows both patient and camera motion, automatically corrects for changes to the internal camera parameters (focal length, focus, aperture, etc.) and requires only a single image. <s> BIB002 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> The problem which must be solved to make realtime enhanced reality visualization possible is basically the camera calibration problem. The relationship between the coordinate frames of the patient, the patient’s internal anatomy scans and the image plane of the camera observing the patient must be established. This paper presents a new approach to finding this relationship and develops a system for performing enhanced reality visualization. Given the locations of a few fiducials our method is fully automatic, runs in nearly real-time, is accurate to a fraction of a pixel, allows both patient and camera motion, automatically corrects for changes to the internal camera parameters (focal length, focus, aperture, etc.) and requires only a single video image. <s> BIB003 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> We present a color-video-based augmented reality (AR) system that is designed to be self-tracking that is, it requires no separate tracking subsystem. Rather, tracking is performed strictly from the video images acquired through the lens of the camera also used to view the real world. The methods for tracking are rooted in prior research in photogrammetry and computer vision. This approach to tracking for AR systems enables a variety of new applications in assembly guidance that are not feasible with current AR technology. Our initial application is in aircraft manufacturing. We outline our approaches to feature detection, correspondence, pose determination, and system calibration. The results obtained thus far are summarized along with the problems we encountered. <s> BIB004 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> Accurate registration between real and virtual objects is crucial for augmented reality applications. Existing tracking methods are individually inadequate: magnetic trackers are inaccurate, mechanical trackers are cumbersome, and vision-based trackers are computationally problematic. We present a hybrid tracking method that combines the accuracy of vision-based tracking with the robustness of magnetic tracking without compromising real-time performance or usability. We demonstrate excellent registration in three sample applications. CR <s> BIB005 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality. <s> BIB006 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> Although Augmented Reality technology was first developed over forty years ago, there has been little survey work giving an overview of recent research in the field. This paper reviews the ten-year development of the work presented at the ISMAR conference and its predecessors with a particular focus on tracking, interaction and display research. It provides a roadmap for future augmented reality research which will be of great value to this relatively young field, and also for helping researchers decide which topics should be explored when they are beginning their own studies in the area. <s> BIB007 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> We are on the verge of ubiquitously adopting Augmented Reality (AR) technologies to enhance our percep- tion and help us see, hear, and feel our environments in new and enriched ways. AR will support us in fields such as education, maintenance, design and reconnaissance, to name but a few. This paper describes the field of AR, including a brief definition and development history, the enabling technologies and their characteristics. It surveys the state of the art by reviewing some recent applications of AR technology as well as some known limitations regarding human factors in the use of AR systems that developers will need to overcome. <s> BIB008 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and problems encountered when building some Augmented Reality applications. It surveys mobile augmented reality systems challenges and requirements for successful mobile systems. This paper summarizes the current applications of Augmented Reality and speculates on future applications and where current research will lead Augmented Reality's development. Challenges augmented reality is facing in each of these applications to go from the laboratories to the industry, as well as the future challenges we can forecast are also discussed in this paper. Section 1 gives an introduction to what Augmented Reality is and the motivations for developing this technology. Section 2 discusses Augmented Reality Technologies with computer vision methods, AR devices, interfaces and systems, and visualization tools. The mobile and wireless systems for Augmented Reality are discussed in Section 3. Four classes of current applications that have been explored are described in Section 4. These applications were chosen as they are the most famous type of applications encountered when researching AR apps. The future of augmented reality and the challenges they will be facing are discussed in Section 5. <s> BIB009 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> This survey paper presents a classification of different challenges and tracking techniquesin the field of augmented reality. The challenges in augmented reality arecategorized into performance challenges, alignment challenges, interaction challenges,mobility/portability challenges and visualization challenges. Augmentedreality tracking techniques are mainly divided into sensor-based tracking, visionbasedtracking and hybrid tracking. The sensor-based tracking is further dividedinto optical tracking, magnetic tracking, acoustic tracking, inertial tracking or anycombination of these to form hybrid sensors tracking. Similarly, the vision-basedtracking is divided into marker-based tracking and markerless tracking. Eachtracking technique has its advantages and limitations. Hybrid tracking provides arobust and accurate tracking but it involves financial and tehnical difficulties. <s> BIB010 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> In the past two decades, augmented reality (AR) has received a growing amount of attention by researchers in the manufacturing technology community, because AR can be applied to address a wide range of problems throughout the assembly phase in the lifecycle of a product, e.g., planning, design, ergonomics assessment, operation guidance and training. However, to the best of authors’ knowledge, there has not been any comprehensive review of AR-based assembly systems. This paper aims to provide a concise overview of the technical features, characteristics and broad range of applications of AR-based assembly systems published between 1990 and 2015. Among these selected articles, two thirds of them were published between 2005 and 2015, and they are considered as recent pertinent works which will be discussed in detail. In addition, the current limitation factors and future trends in the development will also be discussed. <s> BIB011 </s> Augmented-Reality Survey: from Concept to Application <s> Core technologies and Issues of AR <s> The term Augmented Reality (AR) refers to a set of technologies and devices able to enhance and improve human perception, thus bridging the gap between real and virtual space. Physical and artificial objects are mixed together in a hybrid space where the user can move without constraints. This mediated reality is spread in our everyday life: work, study, training, relaxation, time spent traveling are just some of the moments in which you can use AR applications. This paper aims to provide an overview of current technologies and future trends of augmented reality as well as to describe the main application domains, outlining benefits and open issues. <s> BIB012
The major survey papers BIB010 BIB007 BIB006 BIB008 BIB009 BIB011 BIB012 regarding AR that have been published since 1997 have presented the classifications of the core AR technologies and challenges. Table 1 shows a comparison of the core AR technologies that are presented in seven of the major survey papers. In 1997, Azuma pointed out that the registration and the sensing errors were two major problems regarding the development of AR systems BIB006 . Registration means that the objects in the real and virtual environments are properly aligned with respect to each other. Azuma divided the registration errors into the following two types: static and dynamic. Static errors are the ones that are caused by tracking-system errors, optical distortion, or incorrect viewing parameters, even when the user's viewpoint and the target objects are completely fixed. Alternatively, the dynamic errors occur due to systemic delays. Azuma presumed that the tracking-system delay results from the difference between the moment that the tracking system estimates the position and orientation of the viewpoint and the moment that the image corresponding to that position and orientation is displayed. The communication delays, tracking delays, and scene-generation time all contribute to the end-to-end systemic delay. The end-to-end systemic delay causes the dynamic registration errors only when motion occurs. In BIB006 , Azuma mentioned that the registration problem was far from solved even though a few systems had achieved sound results  BIB001 BIB002 BIB003 BIB004 BIB005 . An accurate registration of virtual objects in the real world requires an accurate tracking of the viewpoint and a sensing of the locations of the target objects. In 2008, Zhou et al. reviewed the previously published conference papers from the ISMAR '02 to the ISMAR '07, as well as those of its forerunner events, the IWAR (International Workshop on Augmented Reality) '98, IWAR '99, ISMR (International Symposium on Mixed Reality) '99, ISMR '01, ISAR (International Symposium on Augmented Reality) '00, and ISAR '01 BIB007 . Based on the results of the review of the previous proceedings, the published papers were divided into two groups. The first group contains the five main research areas, and the second group consists of the emergent research interests; that is, the five main research areas of the first group are "tracking," "interaction," "calibration and registration," "AR applications," and "display," while the topics of the second group are "evaluation/testing," "mobile/wearable AR," "AR authoring," "visualization," "multimodal AR," and "rendering," as shown in the second column of Table 1 . In 2010, Krevelen et al. introduced various types of user interface and interaction through their survey paper including biosensing, gaze tracking, and the haptic interface. Moreover, they explained three further requirements for the mobile AR system that had been proposed by Höllerer and Feiner  , as follows: computational framework, wireless networking, and the technology for data storage and access. An easy development of the AR system can be achieved with a computational framework like the ARToolkit. Also, Krevelen et al. emphasized that the specialization of content and the creation of dynamic content through the use of AR-authoring tools are very important for the commercial success of AR systems. Other survey papers that have been presented since 2011 have introduced new tracking and natural-interaction approaches for which a variety of sensors are used. The interest in mobile portability, natural interfaces, dynamic scene generation, and information management in terms of the content has rapidly increased due to the widespread use of mobile devices. In the authors' research process, the conference sessions from the ISMAR '13 through to the ISMAR '16, as shown in Table 2 , were reviewed to investigate the way that the AR-research areas have been changed in comparison with the pre-2008 research areas. A comparison of Table 1 with Table 2 revealed that the main areas including "tracking," "interaction," "calibration," and "display" have been steadily researched; furthermore, visual SLAM (simultaneous localization and mapping) tracking, realistic rendering with a 3D effect, real-time mobile AR, outdoor AR, and collaboration in AR have become increasingly important topics for the advanced AR-research field.