Mobile Reacher Interface for IntuitiveInformation Navigation
Yuichi Yoshida1, Kento Miyaoku1, Takashi Satou1, and Suguru Higashino1
Nippon Telegraph and Telephone Corporation, NTT Cyber CommunicationsLaboratory Hikari-no-oka 1-1 Yokosuka, Kanagawa, Japan{yoshida.yuichi,miyaoku.kento,satou.takashi,higashino.suguru}@lab.ntt.co.jp
Abstract. We propose Mobile Reacher Interface (MoRIn) that is a newinterface with a visual tag for a mobile device. MoRIn allows users tonavigate sophisticated information structures by making natural handgestures while holding a camera-equipped mobile terminal. The structureof an item of interest is acquired via a visual tag. The user can thennavigate the structure as if he/she is handling physical objects. MoRIngreatly eases the user’s cognitive loads. For instance, the user can alterthe volume of an announcement by twisting terminal as if he/she wereturning a dial. We describe MoRIn, some applications, and the resultsof a preliminary experiment.
1Introduction
Many new services are being created that utilize the user’s mobile device asa display system. The user sees public media and acquires a visual tag whichloads a URL into the terminal which then accesses the corresponding informationsource[1,2].Early services provided just a simple link to go to an advertisement but moresophisticated variants are emerging. The current information structure of tag-based services is a simple pointer which brings the typical menu list. That is,the real world is not linked directly to the cyber world and the user is forcedto turn away from the real world and remember the control operations neededto access the information in the menu list, see Fig.1-(a). This poses a problemsince most terminals provide non-intuitive interaction devices such as buttons.A more eﬀective interface is needed.
Fig. 1. Current and proposed style
We believe that this unduly restricts the services available and preventswidespread adoption. Our approach is to replace the pointer with three struc-tures that mirror the real world: the line, surface, and space, see Fig.1-(b). User
can intuitively navigate the information structure which is indicated by the vi-sual tag as if directly manipulating a real world object with a “Mobile Reacher”.To implement MoRIn, we utilize the terminal’s camera to acquire the visualtag and use the physical movements of the terminal as user input to navigate theinformation structure indicated by the tag. The extraction of physical movementfrom visual ﬁeld processing with the tags has already been reported; six DegreesOf Freedom (DOF) are available. For instance, the research has examined theimpact of overlaying the camera image with the information acquired throughthe tag[3, 4].
Fig. 2. Example of MoRIn
MoRIn permits a wide range of intuitive operations. For instance, the usercan select one of several “pages” as if grabbing a real world page, see Fig.2-(a).Fig.2-(b) shows another example, the user can adjust a parameter, such as audiovolume, by rotating the terminal as if rotating a control dial. MoRIn allows usersto control virtual attributes with real world actions.We constructed a prototype MoRIn system and evaluated its performance.In this paper, we describe MoRIn, several applications, and the results of apreliminary evaluation.
2MoRIn - Mobile reacher interface
2.1Basic operations
Fig. 3. Basic operationsA MoRIn system consists of mobile devices with digital cameras, contentservers, and public media, like a poster, magazine, large screen, holding thevisual tags. Each visual tag is structured to provide content ID and operationID. The operations reﬂect the six DOF as shown in Fig.3. Besides, the morecomplex operations can be composed of these basic operations.
2.2Applications
MoRIn services consist of three layers: (1)content layer, (2)logical operationlayer, and (3) operation layer. Two typical MoRIn services are shown in Fig.4.
Restaurant map The restaurant map shows the physical locations of the restau-rants and their visual tags. Each tag calls up and displays the dishes, orderedby price, oﬀered by that restaurant. The logical operation provided, sink, allowsthe user to ﬂip through the dishes by changing the desired price. The user canlower (raise) the desired price by moving the terminal towards (away from) thetag, see Fig.4-(a).
Automatic dis-paper(dispenser) MoRIn allows a poster to become an automaticdis-paper through the metaphor of the automatic dispenser. The poster showsseveral CD jacket pictures and their corresponding visual tag. The contents layerholds songs. The logical operation layer allows the user to select a song, adjustvolume, and purchase a song. The user can select an album by horizontal slide,select a song by sink, adjust volume by spin, and buy a song by vertical slide asshown in Fig.4-(b).
Fig. 4. Example of applications
3Prototype system
We constructed a prototype terminal that is as small as the current cellularphones and used it to evaluate the performance of MoRIn (Fig.5-(a)). The pro-totype terminal was connected to a PC by a cable in order to process MoRInsoftware operation. Fig.5-(b) shows the ﬂowchart of MoRIn software operation;QR Code was used as the visual tag[2].
Fig. 5. Prototype system
4Experiments and resultsWe asked 3 adult subjects with no previous experience of MoRIn to use theprototype terminal to ﬁnd virtual pages via visual tags as described below. Thetags yielded information structures that consisted of 5, 10, 15, and 20 pagesarranged as a vertical stack; the only operation provided was sink. The visualtags were placed on a wall and on a desk so user’s position was either standing upor sitting down. For each tag in each position, the subjects were shown a numberon the screen of the prototype and then told to “ﬁnd the page that shows thesame number.” In each trial the pages were randomly ordered. Each subjectrepeated this action 10 times and we measured the time it took for the subjectsto ﬁnd the correct page. Figure 6-(a) shows the appearance of the trials andFig.6-(b) shows the mean time taken, the bars in the ﬁgure show the standarddeviation.
Fig. 6. Result of experimentsThe results show that the subjects could page through the information struc-tures with 10 and 15 pages in approximately 3 seconds in both positions. Twosubjects commented that MoRIn could be used in the real world. We note thatsystem performance may be degraded if the camera (currently at the center ofthe terminal) is placed oﬀ-center.
5Conclusion and future workWe proposed MoRIn that was a new interface with a visual tag for mobile devices.MoRIn allows users to employ intuitive hand movements to navigate complexinformation structures and retrieve digital content. We described examples ofMoRIn applications, and a preliminary evaluation of the performance of sink us-ing a prototype system. The results conﬁrm that visual tags, printed on postersand magazines, can realize the MoRIn service. We intend to evaluate the perfor-mance of other basic operations, and the performance of practical applicationsof MoRIn.References
1. Arai, T., Aust, D. and Hudson, S. E.: PaperLink:a technique for hyperlinking fromreal paper to electronic content. Proceedings of the SIGCHI. (1997)327–3342. http://www.qrcode.com3. Kato, H. and Billinghurst, M.:Marker tracking and HMD calibration for a video-based augmented reality conferencing system:In Proc. IEEE International Workshopon Augmented Reality. (1999)125–1334. Rekimoto, J. and Ayatsuka, Y.: CyberCode:Designing Augmented Reality Environ-ments with Visual Tags. Designing Augmented Reality Environments. (2000)