Taller and sleeker smartphone devices are becoming the new norm. More screen space and very responsive touchscreens have made for enjoyable experiences available to us at all times. However, after years of interacting with smaller, portable devices, we still try to use these large smartphones on the go, and do not want to change how, where, and when we interact with them. The older devices were easier to use with one hand, when mobile. Now, with bigger devices, users have trouble accessing all parts of the screen with one hand. We need to recognize the limitations in usability due to these large screens. We must start designing user interfaces that are more conducive to one hand usage, which is the preferred way of interacting with the phone. This paper introduces Adaptive App Design, a design methodology that promotes dynamic and adaptive interfaces for one handed usage. We present a novel method of recognizing which hand the user is interacting with and suggest how to design friendlier interfaces for them by presenting a set of design guidelines for this methodology.
@cite_33 @cite_16 @cite_9 @cite_5 are only few of many studies over several years stating that current interface designs on mobile devices are not suitable for one handed usage. Studies such as @cite_15 @cite_13 determine the functional area of the thumb, which quantifies the struggles that users face when operating these ill-designed interfaces.
@cite_33 @cite_9 @cite_5 @cite_15 @cite_16 @cite_13
2004717738 1983317912 2011130597 2008181407 152129288 1980867236 2064376060 1987220739 2030026498 2128267659 2010376411 2130306162 2137940226 2096147482 2731419753 1982673860 2134512786 1979154252 2004403975 1815774045
We present a predictive model for the functional area of the thumb on a touchscreen surface: the area of the interface reachable by the thumb of the hand that is holding the device. We derive a quadratic formula by analyzing the kinematics of the gripping hand. Model fit is high for the thumb-motion trajectories of 20 participants. The model predicts the functional area for a given 1) surface size, 2) hand size, and 3) position of the index finger on the back of the device. Designers can use this model to ensure that a user interface is suitable for interaction with the thumb. The model can also be used inversely - that is, to infer the grips assumed by a given user interface layout. Three studies of different mobile-device hand postures are presented. The first study measures the performance of postures in Fitts' law tasks using one and two hands, thumbs and index fingers, horizontal and vertical movements, and front- and back-of-device interaction. Results indicate that the index finger performs well on both the front and the back of the device, and that thumb performance on the front of the device is generally worse. Fitts' law models are created and serve as a basis for comparisons. The second study examines the orientation of shapes on the front and back of a mobile device. It shows that participants' expectations of visual feedback for finger movements on the back of a device reverse the direction of their finger movements to favor a ''transparent device'' orientation. The third study examines letter-like gestures made on the front and back of a device. It confirms the performance of the index finger on the front of the device, while showing limitations in the ability for the index finger on the back to perform complex gestures. Taken together, these results provide an empirical foundation upon which new mobile interaction designs can be based. A set of design implications and recommendations are given based directly on the findings presented. We present a mobile user interface which dynamically reformulates the layout based on the touch input pattern of users. By analyzing the touch input, it infers users' physical characteristics such as handedness, finger length, or usage habits, thereby calculates the optimal touch area for the user. The user interface is gradually adapted to each user by automatically rearranging graphic objects such as application icons to the most easy-to-touch positions. To compute the optimal touch area, we designed software architecture and implemented an Android application which analyzes touch input and determines the touch frequency in specific screen areas, the handedness and hand size of users. As proof of concept, this research prototype shows acceptable performance and accuracy. To decide which items should be placed in the optimal touch area, we plan to integrate our machine learning algorithm which prioritizes applications according to the context of users into the proposed system. We present PalmType, which uses palms as interactive keyboards for smart wearable displays, such as Google Glass. PalmType leverages users' innate ability to pinpoint specific areas of their palms and fingers without visual attention (i.e. proprioception), and provides visual feedback via the wearable displays. With wrist-worn sensors and wearable displays, PalmType enables typing without requiring users to hold any devices and does not require visual attention to their hands. We conducted design sessions with 6 participants to see how users map QWERTY layout to their hands based on their proprioception. To evaluate typing performance and preference, we conducted a 12-person user study using Google Glass and Vicon motion tracking system, which showed that PalmType with optimized QWERTY layout is 39 faster than current touchpad-based keyboards. In addition, PalmType is preferred by 92 of the participants. We demonstrate the feasibility of wearable PalmType by building a prototype that uses a wrist-worn array of 15 infrared sensors to detect users' finger position and taps, and provides visual feedback via Google Glass. This paper presents a novel user interface for handheld mobile devices by recognizing hand grip patterns. Particularly, we consider the scenario where the device is provided with an array of capacitive touch sensors underneath the exterior cover. In order to provide the users with intuitive and natural manipulation experience, we use pattern recognition techniques for identifying the users' band grips from the touch sensors. Preliminary user studies suggest that filtering out unintended user hand grip is one of the most important issues to be resolved. We discuss the details of the prototype implementation, as well as engineering challenges for practical deployment. A controlled experiment was performed to evaluate the usability of freehand gestural target selection with different 3D marking menu layouts and target directions. We found that a rectangular layout was faster than an octagonal layout, with no significant increase in errors, and that our right-handed participants preferred to select to the right and forwards in a 3D marking menu. We propose an improved design for 3D marking menus based on our findings. The experimental results also suggest that designers should consider carefully whether or not findings from similar interaction techniques using hands-on devices can be carried over to the design of freehand gestural interaction. Authentication methods can be improved by considering implicit, individual behavioural cues. In particular, verifying users based on typing behaviour has been widely studied with physical keyboards. On mobile touchscreens, the same concepts have been applied with little adaptations so far. This paper presents the first reported study on mobile keystroke biometrics which compares touch-specific features between three different hand postures and evaluation schemes. Based on 20.160 password entries from a study with 28 participants over two weeks, we show that including spatial touch features reduces implicit authentication equal error rates (EER) by 26.4 - 36.8 relative to the previously used temporal features. We also show that authentication works better for some hand postures than others. To improve applicability and usability, we further quantify the influence of common evaluation assumptions: known attacker data, training and testing on data from a single typing session, and fixed hand postures. We show that these practices can lead to overly optimistic evaluations. In consequence, we describe evaluation recommendations, a probabilistic framework to handle unknown hand postures, and ideas for further improvements. As mobile and tangible devices are getting smaller and smaller it is desirable to extend the interaction area to their whole surface area. The HandSense prototype employs capacitive sensors for detecting when it is touched or held against a body part. HandSense is also able to detect in which hand the device is held, and how. The general properties of our approach were confirmed by a user study. HandSense was able to correctly classify over 80 percent of all touches, discriminating six different ways of touching the device (hold left right, pick up left right, pick up at top bottom). This information can be used to implement or enhance implicit and explicit interaction with mobile phones and other tangible user interfaces. For example, graphical user interfaces can be adjusted to the user's handedness. Bare hand pointing on touch screens both benefits and suffers from the nature of direct input. This work explores techniques to overcome its limitations. Our goal is to design interaction tools allowing pixel level pointing in a fast and efficient manner. Based on several cycles of iterative design and testing, we propose two techniques: Cross-Keys that uses discrete taps on virtual keys integrated with a crosshair cursor, and an analog Precision-Handle that uses a leverage (gain) effect to amplify movement precision from the user's finger tip to the end cursor. We conducted a formal experiment with these two techniques, in addition to the previously known Zoom-Pointing and Take-Off as baseline anchors. Both subjective and performance measurements indicate that Precision-Handle and Cross-Keys complement existing techniques for touch screen interaction. Mobile devices gain increasing computational power and storage capabilities, and there are already mobile phones that can show movies, act as digital music players and offer full-scale web browsing. The bottleneck for information flow is however limited by the inefficient communication channel between the user and the small device. The small mobile phone form factor has proven to be surprisingly difficult to overcome and limited text entry capabilities are in effect crippling mobile devices’ use experience. The desktop keyboard is too large for mobile phones, and the keypad too limited. In recent years, advanced mobile phones have come equipped with touch-screens that enable new text entry solutions. This dissertation explores how software keyboards on touch-screens can be improved to provide an efficient and practical text and command entry experience on mobile devices. The central hypothesis is that it is possible to combine three elements: software keyboard, language redundancy and pattern recognition, and create new effective interfaces for text entry and control. These are collectively called “shape writing” interfaces. Words form shapes on the software keyboard layout. Users write words by articulating the shapes for words on the software keyboard. Two classes of shape writing interfaces are developed and analyzed: discrete and continuous shape writing. The former recognizes users’ pen or finger tapping motion as discrete patterns on the touch-screen. The latter recognizes users’ continuous motion patterns. Experimental results show that novice users can write text with an average entry rate of 25 wpm and an error rate of 1 after 35 minutes of practice. An accelerated novice learning experiment shows that users can exactly copy a single well-practiced phrase with an average entry rate of 46.5 wpm, with individual phrase entry rate measurements up to 99 wpm. When used as a control interface, users can send commands to applications 1.6 times faster than using de-facto standard linear pull-down menus. Visual command preview leads to significantly less errors and shorter gestures for unpracticed commands. Taken together, the quantitative results show that shape writing is among the fastest mobile interfaces for text entry and control, both initially and after practice, that are currently known. We study the performance and user experience of two popular mainstream mobile text entry methods: the Smart Touch Keyboard (STK) and the Smart Gesture Keyboard (SGK). Our first study is a lab-based ten-session text entry experiment. In our second study we use a new text entry evaluation methodology based on the experience sampling method (ESM). In the ESM study, participants installed an Android app on their own mobile phones that periodically sampled their text entry performance and user experience amid their everyday activities for four weeks. The studies show that text can be entered at an average speed of 28 to 39 WPM, depending on the method and the user's experience, with 1.0 to 3.6 character error rates remaining. Error rates of touchscreen input, particularly with SGK, are a major challenge; and reducing out-of-vocabulary errors is particularly important. Both SGK and STK have strengths, weaknesses, and different individual awareness and preferences. Two-thumb touch typing in a focused setting is particularly effective on STK, whereas one-handed SGK typing with the thumb is particularly effective in more mobile situations. When exposed to both, users tend to migrate from STK to SGK. We also conclude that studies in the lab and in the wild can both be informative to reveal different aspects of keyboard experience, but used in conjunction is more reliable in comprehensively assessing input technologies of current and future generations. This paper reports on the design and use of tactile user interfaces embedded within or wrapped around the devices that they control. We discuss three different interaction prototypes that we built These interfaces were embedded onto two handheld devices of dramatically different form factors. We describe the design and implementation challenges, and user feedback and reactions to these prototypes. Implications for future design in the area of manipulative or haptic user interfaces are highlighted. Direct use of the hand as an input device is an attractive method for providing natural human-computer interaction (HCI). Currently, the only technology that satisfies the advanced requirements of hand-based input for HCI is glove-based sensing. This technology, however, has several drawbacks including that it hinders the ease and naturalness with which the user can interact with the computer-controlled environment, and it requires long calibration and setup procedures. Computer vision (CV) has the potential to provide more natural, non-contact solutions. As a result, there have been considerable research efforts to use the hand as an input device for HCI. In particular, two types of research directions have emerged. One is based on gesture classification and aims to extract high-level abstract information corresponding to motion patterns or postures of the hand. The second is based on pose estimation systems and aims to capture the real 3D motion of the hand. This paper presents a literature review on the latter research direction, which is a very challenging problem in the context of HCI. We explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of asymmetric and symmetric interactions, including fast targeting and navigation to all parts of a large display from the comfort of a desk and chair, as well as techniques that exploit the ability of the vision-based hand tracking system to provide multi-finger identification and full 2D hand segmentation. We also posit a design that allows for handling multiple concurrent users. Eyes-free input is desirable for ubiquitous computing, since interacting with mobile and wearable devices often competes for visual attention with other devices and tasks. In this paper, we explore eyes-free typing on a touchpad using one thumb, wherein a user taps on an imaginary QWERTY keyboard while receiving text feedback on a separate screen. Our hypothesis is that users can transfer their typing ability obtained from visible keyboards to eyes-free use. We propose two statistical decoding algorithms to infer users’ eyes-free input: the absolute algorithm and the relative algorithm. The absolute algorithm infers user input based on the absolute position of touch endpoints, while the relative algorithm infers based on the vectors between successive touch endpoints. Evaluation results showed users could achieve satisfying performance with both algorithms. Text entry rate was 17-23 WPM (words per minute) depending on the algorithm used. In comparison, a baseline cursor-based text entry method yielded only 7.66 WPM. In conclusion, our research demonstrates for the first time the feasibility of thumb-based eyes-free typing, which provides a new possibility for text entry on ubiquitous computing platforms such as smart TVs and HMDs. The input vocabulary for touch-screen interaction on handhelds is dramatically limited, especially when the thumb must be used. To enrich that vocabulary we propose to discriminate, among thumb gestures, those we call MicroRolls, characterized by zero tangential velocity of the skin relative to the screen surface. Combining four categories of thumb gestures, Drags, Swipes, Rubbings and MicroRolls, with other classification dimensions, we show that at least 16 elemental gestures can be automatically recognized. We also report the results of two experiments showing that the roll vs. slide distinction facilitates thumb input in a realistic copy and paste task, relative to existing interaction techniques. Text entry user interfaces have been a bottleneck of non traditional computing devices. One of the promising methods is the virtual keyboard on touch screens. Various layouts have been manually designed to replace the dominant QWERTY layout. This paper presents two computerized quantitative design techniques to search for the optimal virtual keyboard. The first technique simulated the dynamics of a keyboard with “digraph springs” between keys, which produced a “Hooke’s” keyboard with 41.6 wpm performance. The second technique used a Metropolis random walk algorithm guided by a “Fitts energy” objective function, which produced a “Metropolis” keyboard with 43.1 wpm performance. The paper also models and evaluates the perfo rmance of four existing keyboard layouts. We corrected erroneous estimates in the literature and predicted the performance of QWERTY, CHUBON, FITALY, OPTI to be in the neighborhood of 30, 33, 36 and 38 wpm respectively. Our best design was 40 faster than QWERTY and 10 faster than OPTI, illustrating the advantage of quantitative user interface design techniques based on models of human performance over traditional trial and error designs guided by heuristics. We propose a vision of a grasp-based interaction system where users' intentions are inferred by the way they hold and interact with a device. In this paper we specifically discuss the Bar of Soap, a multi-function handheld prototype that uses grasp-based interactions to switch between modes. This prototype relies on the hypothesis that users share a set of stereotyped grasps associated with common multi-function modes. We show that using common machine learning techniques our device can reliably distinguish five separate classes based on the users' grasps. While this interaction is currently implemented in a multi-function handheld, we anticipate the existence of many scenarios where grasp recognition could provide a more intuitive or useful interface. We present the design and evaluation of TapTap and MagStick, two thumb interaction techniques for target acquisition on mobile devices with small touch-screens. These two techniques address all the issues raised by the selection of targets with the thumb on small tactile screens: screen accessibility, visual occlusion and accuracy. A controlled experiment shows that TapTap and MagStick allow the selection of targets in all areas of the screen in a fast and accurate way. They were found to be faster than four previous techniques except Direct Touch which, although faster, is too error prone. They also provided the best error rate of all tested techniques. Finally the paper also provides a comprehensive study of various techniques for thumb based touch-screen target selection. Hand motion capture is one of the most important parts of gesture interfaces. Many current approaches to this task generally involve a formidable nonlinear optimization problem in a large search space. Motion capture can be achieved more cost-efficiently when considering the motion constraints of a hand. Although some constraints can be represented as equalities or inequalities, there exist many constraints which cannot be explicitly represented. In this paper, we propose a learning approach to model the hand configuration space directly. The redundancy of the configuration space can be eliminated by finding a lower-dimensional subspace of the original space. Finger motion is modeled in this subspace based on the linear behavior observed in the real motion data collected by a CyberGlove. Employing the constrained motion model, we are able to efficiently capture finger motion from video inputs. Several experiments show that our proposed model is helpful for capturing articulated motion.