3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> In this paper we investigate whether virtual reality (VR) and augmented reality (AR) offer potential for the training of manual skills, such as for assembly tasks, in comparison to conventional media. We present results from experiments that compare assembly completion times for a number of different conditions. We firstly investigate completion times for a task where participants can study an engineering drawing and an assembly plan and then conduct the task. We then investigate the task under various VR conditions and context-free AR. We discuss the relative advantages and limitations of using VR and AR as training media for investigating assembly operations, and we present the results of our experimental work. <s> BIB001 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Abstract The MagicBook is a Mixed Reality interface that uses a real book to seamlessly transport users between Reality and Virtuality. A vision-based tracking method is used to overlay virtual models on real book pages, creating an Augmented Reality (AR) scene. When users see an AR scene they are interested in they can fly inside it and experience it as an immersive Virtual Reality (VR). The interface also supports multi-scale collaboration, allowing multiple users to experience the same virtual environment either from an egocentric or an exocentric perspective. In this paper we describe the MagicBook prototype, potential applications and user feedback. <s> BIB002 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies. <s> BIB003 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Foreword. Preface. I. FOUNDATIONS OF 3D USER INTERFACES . 1. Introduction to 3D User Interfaces. What Are 3D User Interfaces? Why 3D User Interfaces? Terminology. Application Areas. Conclusion. 2. 3D User Interfaces: History and Roadmap. History of 3D UIs. Roadmap to 3D UIs. Scope of This Book. Conclusion. II. Hardware Technologies for 3D User Interfaces. 3. 3D User Interface Output Hardware. Introduction. Visual Displays. Auditory Displays. Haptic Displays. Design Guidelines: Choosing Output Devices for 3D User Interfaces. Conclusion. 4. 3D User Interface Input Hardware. Introduction. Desktop Input Devices. Tracking Devices. 3D Mice. Special-Purpose Input Devices. Direct Human Input. Home-Brewed Input Devices. Choosing Input Devices for 3D Interfaces. III. 3D INTERACTION TECHNIQUES. 5. Selection and Manipulation. Introduction. 3D Manipulation Tasks. Manipulation Techniques and Input Devices. Interaction Techniques for 3D Manipulation. Design Guidelines. 6. Travel. Introduction. 3D Travel Tasks. Travel Techniques. Design Guidelines. 7. Wayfinding. Introduction. Theoretical Foundations. User-Centered Wayfinding Support. Environment-Centered Wayfinding Support. Evaluating Wayfinding Aids. Design Guidelines. Conclusion. 8. System Control. Introduction. Classification. Graphical Menus. Voice Commands. Gestural Commands. Tools. Multimodal System Control Techniques. Design Guidelines. Case Study: Mixing System Control Methods. 8.10. Conclusion. 9. Symbolic Input. Introduction. Symbolic Input Tasks. Symbolic Input Techniques. Design Guidelines. Beyond Text and Number Entry. IV. DESIGNING AND DEVELOPING 3D USER INTERFACES. 10. Strategies for Designing and Developing 3D User Interfaces. Introduction. Designing for Humans. Inventing 3D User Interfaces. Design Guidelines. 11. Evaluation of 3D User Interfaces. Introduction. Background. Evaluation Metrics for 3D Interfaces. Distinctive Characteristics of 3D Interface Evaluation. Classification of 3D Evaluation Methods. Two Multimethod Approaches. Guidelines for 3D Interface Evaluation. V. THE FUTURE OF 3D USER INTERFACES. 12. Beyond Virtual: 3D User Interfaces for the Real World. Introduction. AR Interfaces as 3D Data Browsers. 3D Augmented Reality Interfaces. Augmented Surfaces and Tangible Interfaces. Tangible AR Interfaces. Agents in AR. Transitional AR-VR Interfaces. Conclusion. 13. The Future of 3D User Interfaces. Questions about 3D UI Technology. Questions about 3D Interaction Techniques. Questions about 3D UI Design and Development. Questions about 3D UI Evaluation. Million-Dollar Questions. Appendix A: Quick Reference Guide to 3D User Interface Mathematics. Scalars. Vectors. Points. Matrices. Quaternions. Bibliography. Index. <s> BIB004 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Objective To evaluate the history and current knowledge of computer-augmented reality in the field of surgery and its potential goals in education, surgeon training, and patient treatment. Data Sources National Library of Medicine's database and additional library searches. Study Selection Only articles suited to surgical sciences with a well-defined aim of study, methodology, and precise description of outcome were included. Data Synthesis Augmented reality is an effective tool in executing surgical procedures requiring low-performance surgical dexterity; it remains a science determined mainly by stereotactic registration and ergonomics. Strong evidence was found that it is an effective teaching tool for training residents. Weaker evidence was found to suggest a significant influence on surgical outcome, both morbidity and mortality. No evidence of cost-effectiveness was found. Conclusions Augmented reality is a new approach in executing detailed surgical operations. Although its application is in a preliminary stage, further research is needed to evaluate its long-term clinical impact on patients, surgeons, and hospital administrators. Its widespread use and the universal transfer of such technology remains limited until there is a better understanding of registration and ergonomics. <s> BIB005 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> A digital tabletop offers several advantages over other groupware form factors for collaborative applications. However, users of a tabletop system do not share a common perspective for the display of information: what is presented right side up to one participant is upside down for another. In this paper, we survey five different rotation and translation techniques for objects displayed on a direct touch digital tabletop display. We analyze their suitability for interactive tabletops in light of their respective input and output degrees of freedom, as well as the precision and completeness provided by each. We describe various tradeoffs that arise when considering which, when and where each of these techniques might be most useful. <s> BIB006 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Although Augmented Reality technology was first developed over forty years ago, there has been little survey work giving an overview of recent research in the field. This paper reviews the ten-year development of the work presented at the ISMAR conference and its predecessors with a particular focus on tracking, interaction and display research. It provides a roadmap for future augmented reality research which will be of great value to this relatively young field, and also for helping researchers decide which topics should be explored when they are beginning their own studies in the area. <s> BIB007 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Augmented Reality (AR) research has been conducted for several decades, although until recently most AR applications had simple interaction methods using traditional input devices. AR tracking, display technology and software has progressed to the point where commercial applications can be developed. However there are opportunities to provide new advanced interaction techniques for AR applications. In this paper we describe several interaction methods that can be used to provide a better user experience, including tangible user interaction, multimodal input and mobile interaction. <s> BIB008 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> This paper provides a classification of perceptual issues in augmented reality, created with a visual processing and interpretation pipeline in mind. We organize issues into ones related to the environment, capturing, augmentation, display, and individual user differences. We also illuminate issues associated with more recent platforms such as handhelds or projector-camera systems. Throughout, we describe current approaches to addressing these problems, and suggest directions for future research. <s> BIB009 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> The technical enablers for mobile augmented reality (MAR) are becoming robust enough to allow the development of MAR services that are truly valuable for consumers. Such services would provide a novel interface to the ubiquitous digital information in the physical world, hence serving in great variety of contexts and everyday human activities. To ensure the acceptance and success of future MAR services, their development should be based on knowledge about potential end users' expectations and requirements. We conducted 16 semi-structured interview sessions with 28 participants in shopping centres, which can be considered as a fruitful context for MAR services. We aimed to elicit new knowledge about (1) the characteristics of the expected user experience and (2) central user requirements related to MAR in such a context. From a pragmatic viewpoint, the participants expected MAR services to catalyse their sense of efficiency, empower them with novel context-sensitive and proactive functionalities and raise their awareness of the information related to their surroundings with an intuitive interface. Emotionally, MAR services were expected to offer stimulating and pleasant experiences, such as playfulness, inspiration, liveliness, collectivity and surprise. The user experience categories and user requirements that were identified can serve as targets for the design of user experience of future MAR services. <s> BIB010 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Create amazing mobile augmented reality apps with junaio, Layar, and Wikitude!Professional Augmented Reality Browsers for Smartphones guides you through creating your own augmented reality apps for the iPhone, Android, Symbian, and bada platforms, featuring fully workable and downloadable source code. You will learn important techniques through hands-on applications, and you will build on those skills as the book progresses.Professional Augmented Reality Browsers for Smartphones:Describes how to use the latitude/longitude coordinate system to build location-aware solutions and tells where to get POIs for your own augmented reality applicationsDetails the leading augmented reality platforms and highlights the best applicationsCovers development for the leading augmented reality browser platforms: Wikitude, Layar, and junaioShows how to build cross-platform location-aware content (Android, iPhone, Symbian, and bada) to display POIs directly in camera viewIncludes tutorials for building 2D and 3D content, storing content in databases, and triggering actions when users reach specific locationswrox.comProgrammer ForumsJoin our Programmer to Programmer forums to ask and answer programming questions about this book, join discussions on the hottest topics in the industry, and connect with fellow programmers from around the world.Code DownloadsTake advantage of free code samples from this book, as well as code samples from hundreds of other books, all ready to use.Read MoreFind articles, ebooks, sample chapters, and tables of contents for hundreds of books, and more reference resources on programming topics that matter to you.Wrox Professional guides are planned and written by working programmers to meet the real-world needs of programmers, developers, and IT professionals. Focused and relevant, they address the issues technology professionals face every day. They provide examples, practical solutions, and expert education in new technologies, all designed to help programmers do a better job. <s> BIB011 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Traditionally, the field of Human Computer Interaction (HCI) was primarily concerned with designing and investigating interfaces between humans and machines. The primary concern of Surface Computing is also still about designing better interfaces to information. However, with recent technological advances, the concept of "enhancing", "augmenting" or even "re-designing" humans themselves is becoming a very feasible and serious topic of scientific research as well as engineering development. "Augmented Human" is a term that I use to refer to this overall research direction. Augmented Human introduces a fundamental paradigm shift in HCI: from human-computer-interaction to human-computer-integration. In this talk, I will discuss rich possibilities and distinct challenges in enhancing human abilities. I will introduce recent projects conducted by our group including design and applications of wearable eye sensing for augmenting our perception and memory abilities, design of flying cameras as our external eyes, a home appliance that can increase your happiness, an organic physical wall/window that dynamically mediates the environment, and an immersive human-human communication called "JackIn". <s> BIB012 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Abstract Mobile augmented reality (AR) applications can provide just-in-time information based on the user's preferences and context and thus improve the tourist experience. Due to various problems, the potential of this technology has yet to be fully exploited. In this paper, we present the design, implementation and evaluation of a mobile AR application for historical Oslo that aims to bring history to life by providing historical pictures of a location, depending on the direction in which the camera is pointing. This application can run offline and is designed as a generic framework where a similar application for a new city can be created by simply replacing the city-specific database. <s> BIB013 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Purpose ::: During autopsy, forensic pathologists today mostly rely on visible indication, tactile perception and experience to determine the cause of death. Although computed tomography (CT) data is often available for the bodies under examination, these data are rarely used due to the lack of radiological workstations in the pathological suite. The data may prevent the forensic pathologist from damaging evidence by allowing him to associate, for example, external wounds to internal injuries. To facilitate this, we propose a new multimodal approach for intuitive visualization of forensic data and evaluate its feasibility. <s> BIB014 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Abstract Augmented Reality (AR) is an innovative human-machine interaction that overlays virtual components on a real world environment with many potential applications in different fields, ranging from training activities to everyday life (entertainment, head-up display in car windscreens, etc.). The capability to provide the user of the needed information about a process or a procedure directly on the work environment, is the key factor for considering AR as an effective tool to be also used in Through-life Engineering Services (TES). Many experimental implementations have been made by industries and academic institutions in this research area: applications in remote maintenance, diagnostics, non-destructive testing, repairing and setup activities represent the most meaningful examples carried out in the last few years. These applications have concerned different working environments such as aerospace, railway, industrial plants, machine tools, military equipment, underground pipes, civil constructions, etc. The keynote paper will provide a comprehensive survey by reviewing some recent applications in these areas, emphasizing potential advantages, limits and drawbacks, as well as open issues which could represent new challenges for the future. <s> BIB015 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Lately, mobile augmented reality (AR) has become very popular and is used for many commercial and product promotional activities. However, in almost all mobile AR applications, the user only views annotated information or the preset motion of the virtual object in an AR environment and is unable to interact with the virtual objects as if he/she were interacting with real objects in the real environment. In this paper, in an attempt to realize enhanced intuitive and realistic object manipulation in the mobile AR environment, we propose an integrated view-input AR interaction method, which integrates user device manipulation and virtual object manipulation. The method enables the user to hold a 3D virtual object by touching the displayed object on the 2D touch screen of a mobile device, and to move and rotate the object by moving and rotating the mobile device while viewing the held object by way of the 2D screen of the mobile device. Based on this concept, we implemented three types of integrated methods, namely the Rod, Center, and Touch methods, and conducted a user study to investigate the baseline performance metrics of the proposed method on an AR object manipulation task. The Rod method achieved the highest success rate (91%). Participants' feedback indicated that this is because the Rod method is the most natural, and evoked a fixed mental model that is conceivable in the real environment. These results indicated that visualizing the manipulation point on the screen and restricting the user's interactivity with virtual objects from the user's position of view based on a conceivable mental model would be able to aid the user to achieve precise manipulation. <s> BIB016 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Abstract Handheld Augmented Reality (HAR) has the potential to introduce Augmented Reality (AR) to large audiences due to the widespread use of suitable handheld devices. However, many of the current HAR systems are not considered very practical and they do not fully answer to the needs of the users. One of the challenging areas in HAR is the in-situ AR content creation where the correct and accurate positioning of virtual objects to the real world is fundamental. Due to the hardware limitations of handheld devices and possible restrictions in the environment, the correct 3D positioning of objects can be difficult to achieve we are unable to use AR markers or correctly map the 3D structure of the environment. We present SlidAR, a 3D positioning for Simultaneous Localization And Mapping (SLAM) based HAR systems. SlidAR utilizes 3D ray-casting and epipolar geometry for virtual object positioning. It does not require a perfect 3D reconstruction of the environment nor any virtual depth cues. We have conducted a user experiment to evaluate the efficiency of SlidAR method against an existing device-centric positioning method that we call HoldAR. Results showed that SlidAR was significantly faster, required significantly less device movement, and also got significantly better subjective evaluation from the test participants. SlidAR also had higher positioning accuracy, although not significantly. <s> BIB017 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Hand-held smart devices are equipped with powerful processing units, high resolution screens and cameras, that in combination makes them suitable for video see-through Augmented Reality. Many Augmented Reality applications require interaction, such as selection and 3D pose manipulation. One way to perform intuitive, high precision 3D pose manipulation is by direct or indirect mapping of device movement. There are two approaches to device movement interaction; one fixes the virtual object to the device, which therefore becomes the pivot point for the object, thus makes it difficult to rotate without translate. The second approach avoids latter issue by considering rotation and translation separately, relative to the object's center point. The result of this is that the object instead moves out of view for yaw and pitch rotations. In this paper we study these two techniques and compare them with a modification where user perspective rendering is used to solve the rotation issues. The study showed that the modification improves speed as well as both perceived control and intuitiveness among the subjects. <s> BIB018 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Augmented Reality (AR) supports natural interaction in physical and virtual worlds, so it has recently given rise to a number of novel interaction modalities. This paper presents a method for using hand-gestures with speech input for multimodal interaction in AR. It focuses on providing an intuitive AR environment which supports natural interaction with virtual objects while sustaining accessible real tasks and interaction mechanisms. The paper reviews previous multimodal interfaces and describes recent studies in AR that employ gesture and speech inputs for multimodal input. It describes an implementation of gesture interaction with speech input in AR for virtual object manipulation. Finally, the paper presents a user evaluation of the technique, showing that it can be used to improve the interaction between virtual and physical elements in an AR environment. <s> BIB019 </s> 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review <s> II. HANDHELD MOBILE AR INTERFACE <s> Inspection tasks focus on observation of the environment and are required in many industrial domains. Inspectors usually execute these tasks by using a guide such as a paper manual, and directly observing the environment. The effort required to match the information in a guide with the information in an environment and the constant gaze shifts required between the two can severely lower the work efficiency of inspector in performing his/her tasks. Augmented reality (AR) allows the information in a guide to be overlaid directly on an environment. This can decrease the amount of effort required for information matching, thus increasing work efficiency. AR guides on head-mounted displays (HMDs) have been shown to increase efficiency. Handheld AR (HAR) is not as efficient as HMD-AR in terms of manipulability, but is more practical and features better information input and sharing capabilities. In this study, we compared two handheld guides: an AR interface that shows 3D registered annotations, that is, annotations having a fixed 3D position in the AR environment, and a non-AR picture interface that displays non-registered annotations on static images. We focused on inspection tasks that involve high information density and require the user to move, as well as to perform several viewpoint alignments. The results of our comparative evaluation showed that use of the AR interface resulted in lower task completion times, fewer errors, fewer gaze shifts, and a lower subjective workload. We are the first to present findings of a comparative study of an HAR and a picture interface when used in tasks that require the user to move and execute viewpoint alignments, focusing only on direct observation. Our findings can be useful for AR practitioners and psychology researchers. <s> BIB020
Handheld mobile interaction has been upheld as the new direction in human-computer interaction (HCI) field to enhance human ability in HCI using AR BIB012 . Due to this concern, interaction in AR becomes the main topic and research interest among researchers. One of the important aspects is creating appropriate interaction techniques for AR applications that allow end users to interact with virtual content in an intuitive way BIB007 . In AR, there are several categories of interaction technique and basically, interaction can be divided into three major parts: 1) tangible user interaction, 2) multimodal input BIB019 and 3) mobile interaction BIB002 . Since interaction is the wide topic, this paper focuses on handheld mobile interaction and entail 3D interaction in AR due to the rapid advent of the application on cell phones, and now smartphones and tablets, bringing AR almost to the mainstream BIB009 . In this paper, handheld mobile AR is our main focus. Handheld mobile AR represents the AR system that based on handheld mobile devices especially smartphone or tablet which are the popular and daily communication devices nowadays. Recent available handheld mobile AR applications promote advanced tools such as Wikitude [16] , Layar BIB010 and Junaio BIB011 that show the potential of handheld mobile AR interface in consumer markets. In several studies reviewed in this paper, researchers using the word ''mobile'' AR to represent mobile/smartphone or tablet-based AR in their studies while some other studies using the word ''handheld'' AR to represent mobile/smartphone or tabletbased AR system. In order to avoid any confusion of the main concern of this paper, we combining the term ''handheld'' and ''mobile'' as ''handheld mobile'' AR to represent mobile/smartphone or tablet based AR differentiate from ''mobile'' AR that not only represents mobile/smartphone based AR but also meaning head-mounted displays (HMD) based AR system such as Google glasses BIB008 ,  which not the main scope of this paper. We also avoiding for using ''handheld'' AR which not only represents mobile/ smartphone-based AR since handheld devices also bringing the meaning of handheld entertaining devices such as Vive controller used in gamification field which is outside the scope of our discussion. In handheld mobile AR, there are a number of important differences between using a handheld mobile AR interface and a traditional desktop/tabletop AR interface, including: 1) different input options (no mouse or keyboard) BIB008 , 2) limited screen size BIB008 , and 3) limited activity time due to the battery operation hour of handheld mobile devices. Meanwhile, compared to the HMDs-based AR system, in an AR application on a phone/tablet, the display is handheld rather than head-worn, and the display and input components are connected within a single device mean that interface metaphors developed for desktop and HMDs-based systems may not be appropriate for those handheld mobile AR BIB008 . Therefore, this paper devoted to handheld mobile devices due to their robustness than HMDs to enter the mass market. Furthermore, based on the analysis done by  , for the papers published within years 2005 to 2014, the numbers of researches using HMDs and desktop/tabletop started to decrease replaced by the researches done using handheld mobile displays because of the popularity and ubiquity of smartphones. Consequently, this trend leads to researches and studies being conducted to explore the full potential of handheld mobile AR. On the basis of these concepts, many techniques have been introduced based on handheld mobile AR interaction that allows the user to interact with the virtual object in real environments. In handheld mobile AR, the 3D interaction task was discussed in many previous works BIB006 - BIB017 that are based on fundamental 3D object manipulation tasks including 3D object selection, translation, and rotation BIB004 . As stated in BIB008 , most of the earlier AR systems were used to view virtual models in application domains  ,  while only little support for creating or modifying the AR content. As stated in the AR definition, interaction in 3D is one of the basic elements of the AR system and interaction in 3D are always involving 3D object manipulation tasks due to its practicability and functionality potential. Due to the huge market potential for handheld mobile devices, handheld mobile AR has played a predominant role in present researches  , BIB007 . It has found application in tourist industries BIB013 , maintenance  and task support BIB020 , education  , and medical practices BIB005 - BIB014 respectively. An essential part of AR applications is the spatially consistent alignment of virtual and real objects BIB003 that required accurate and efficient 3D object manipulation. It is crucially important when AR potentially has been explored to support the real-world assembly tasks such as BIB001 - BIB015 . Thus, this paper focusing on 3D object manipulation techniques on handheld mobile AR while HMDs-based AR and desktop/tabletop AR are included only as background discussion since many of the researches in this context are related with the phylogeny of 3D object manipulation techniques within handheld mobile AR. Based on 3D object manipulation, the interaction technique in handheld mobile AR can be categorized into 1) touchbased interaction, 2) mid-air gestures-based interaction and 3) device-based interaction BIB017 , BIB016 , BIB018 as shown in Fig. 2 . Each type of interaction has its own limitations and strengths. The implementation of the 3D object manipulation of virtual objects in handheld mobile AR has been complicated by the fact that almost all handheld mobile AR systems are implemented on smartphones and tablet by means of the tracker (used to track the AR marker) is the built-in camera which also has been used as the rendering system associated with the display screen, all together on the same platform(smartphone or tablet). As a rough definition of each technique category, touchbased interaction is an interaction category that involves all uses of on-screen touch inputs that are performed by fingertips for the manipulation of 3D objects. Mid-air gesturesbased interaction category includes all techniques by means of tracking the bare hands or finger gestures as inputs that allow the manipulation of 3D objects. Device-based interaction category includes techniques that enable the handled mobile device's physical attributes to be tracked for 3D object manipulation (see Fig. 2 ). Users rotate, tilt, skew and move the mobile device itself as inputs by which to manipulate the 3D object. In the next section, we describe the three (3) main interaction technique categories with their phylogeny comprehensively and summarize the issues and problems consisted within each category in an integrated table. In order to describe each category in more affluent manner due to their phylogeny, the relevant techniques in virtual environment and tabletop displays also been discussed since many of the interaction techniques proposed and used in large-display based AR systems with assistance of head-worn devices until handheld mobile AR (small displays) have formerly come from the techniques used previously in those systems. These techniques had been modified, improved or even applied directly into the AR system based on their suitability and applicability. Therefore, we firstly discussed each category from a macroscopic perspective (including techniques used in the virtual environment and large-display AR systems) and narrowing it into handheld mobile AR based on 3D object manipulation techniques in a synchronized way.