We present and evaluate an approach for human-in-the-loop specification of shape reconstruction with annotations for basic robot-object interactions. Our method is based on the idea of model annotation: the addition of simple cues to an underlying object model to specify shape and delineate a simple task. The goal is to explore reducing the complexity of CAD-like interfaces so that novice users can quickly recover an object's shape and describe a manipulation task that is then carried out by a robot. The object modeling and interaction annotation capabilities are tested with a user study and compared against results obtained using existing approaches. The approach has been analyzed using a variety of shape comparison, grasping, and manipulation metrics, and tested with the PR2 robot platform, where it was shown to be successful.
On the other hand, there has been a lot of effort in developing methods for reducing the cognitive load on the user in 2D-GUIs. In @cite_77 , simple constraining of motions was effective in offloading some of the cognitive effort. The familiarity and widespread use of smartphones and tablets has also brought attention to touch-based control. In @cite_78 , a simple touch-based interface was aimed at reducing operator fatigue.
@cite_77 @cite_78
2160641905 2093956463 2012976444 2170678421 2167495023 2138207021 2100557470 2095416579 2017919391 2592582799 2009127721 2030026498 2122115534 2344260694 2088078109 1983885373 2963199973 2136317921 2421020186 2067977053
Given the ease that humans have with using a keyboard and mouse in typical, non-colocated computer interaction, many studies have investigated the value of colocating the visual field and motor workspaces using immersive display modalities. Significant understanding has been gained by previous work comparing physical tasks against virtual tasks, visuomotor colocation versus non-colocation, and even visuomotor rotational misalignments in virtual environments (VEs). However, few studies have explored all of these paradigms in context with each other, and it is difficult to perform interstudy comparisons because of the variation in tested motor tasks. Therefore, using a stereoscopic fish tank display setup, the goal for the current study was to characterize human performance of a 3D Fitts' point-to-point reaching task using a stylus-based haptic interface in the physical, colocated non-colocated, and rotated VE visualization conditions. Five performance measures---throughput, efficiency, initial movement error, corrective movements, and peak velocity---were measured and used to evaluate task performance. These measures were studied in 22 subjects (11 male, 11 female, ages 20--32) performing a 3D variant of Fitts' serial task under 10 task conditions: physical, colocated VE, non-colocated VE, and rotated VEs from 45---315° in 45° increments. Hypotheses: All performance measures in the colocated VE were expected to reflect significantly reduced task performance over the real condition, but also reflect increased performance over the non-colocated VE condition. For rotational misalignments, all performance measures were expected to reflect the highest performance at 0°, reduce to the lowest performance at 90°, and rise again to a local maximum at 180° (symmetric about 0°). Results: All performance measures showed that the colocated VE condition resulted in significantly lower task performance than the physical condition and higher mean performance than the non-colocated VE condition, but the difference was not statistically significant. Also, rotation misalignments showed that task performance was mostly reduced to a minimum at 90°, 135°, and 225°. We conclude that colocated VEs may not significantly improve point-to-point reaching performance over non-colocated VEs. Also, visual rotations of ±45° affected throughput, efficiency, peak velocity, and initial movement error, but the number of corrective movements were not affected until ± 90°. This paper presents a set of interaction techniques for hands-free multi-scale navigation through virtual environments. We believe that hands-free navigation, unlike the majority of navigation techniques based on hand motions, has the greatest potential for maximizing the interactivity of virtual environments since navigation modes are offloaded from modal hand gestures to more direct motions of the feet and torso. Not only are the users’ hands freed to perform tasks such as modeling, notetaking and object manipulation, but we also believe that foot and torso movements may inherently be more natural for some navigation tasks. The particular interactions that we developed include a leaning technique for moving small and medium distances, a foot-gesture controlled Step WIM that acts as a floor map for moving larger distances, and a viewing technique that enables a user to view a full 360 degrees in only a three-walled semi-immersive environment by subtly amplifying the mapping between their torso rotation and the virtual world. We formatively designed and evaluated our techniques in existing projects related to archaeological reconstructions, free-form modeling, and interior design. In each case, our informal observations have indicated that motions such as walking and leaning are both appropriate for navigation and are effective in cognitively simplifying complex virtual environment interactions since functionality is more evenly distributed across the body. Powered wheelchair users often struggle to drive safely and effectively and, in more critical cases, can only get around when accompanied by an assistant. To address these issues, we propose a collaborative control mechanism that assists users as and when they require help. The system uses a multiple-hypothesis method to predict the driver's intentions and, if necessary, adjusts the control signals to achieve the desired goal safely. The main emphasis of this paper is on a comprehensive evaluation, where we not only look at the system performance but also, perhaps more importantly, characterize the user performance in an experiment that combines eye tracking with a secondary task. Without assistance, participants experienced multiple collisions while driving around the predefined route. Conversely, when they were assisted by the collaborative controller, not only did they drive more safely but also they were able to pay less attention to their driving, resulting in a reduced cognitive workload. We discuss the importance of these results and their implications for other applications of shared control, such as brain-machine interfaces, where it could be used to compensate for both the low frequency and the low resolution of the user input. There has been considerable interest in producing grasping platforms using non-invasive, low bandwidth brain computer interfaces(BCIs). Most of this work focuses on low level control of simple hands. Using complex hands improves the versatility of a grasping platform at the cost of increasing its complexity. In order to control more complex hands with these low bandwidth signals, we need to use higher level abstractions. Here, we present a user interface which allows the user to combine the speed and convenience of offline preplanned grasps with the versatility of an online planner. This system incorporates a database of pre-planned grasps with the ability to refine these grasps using an online planner designed for arbitrarily complex hands. Only four commands are necessary to control the entire grasping pipeline, allowing us to use a low cost, noninvasive commercial BCI device to produce robust grasps that reflect user intent. We demonstrate the efficacy of this system with results from five subjects and present results using this system to grasp unknown objects. User performance with a tabletop display was tested using touch-based and mouse-based interaction in a traditional pointing task. Dependent variables were throughput, error rate, and movement time. In a study with 12 participants, touch had a higher throughput with average of 5.53 bps compared to 3.83 bps for the mouse. Touch also had a lower movement time on average, with block means ranging from 403 ms to 1051 ms vs. 607 ms to 1323 ms with the mouse. Error rates were lower for the mouse at 2.1 , compared to 9.8 for touch. The high error rates using touch were attributed to problems in selecting small targets with the finger. It is argued that, overall, touch input is a preferred and efficient input technique for tabletop displays, but that more research is needed to improve touch selection of small targets. Vision-based interfaces, such as those made popular by the Microsoft Kinect, suffer from the Midas Touch problem: every user motion can be interpreted as an interaction. In response, we developed an algorithm that combines facial features, body pose and motion to approximate a user's intention to interact with the system. We show how this can be used to determine when to pay attention to a user's actions and when to ignore them. To demonstrate the value of our approach, we present results from a 30-person lab study conducted to compare four engagement algorithms in single and multi-user scenarios. We found that combining intention to interact with a 'raise an open hand in front of you' gesture yielded the best results. The latter approach offers a 12 improvement in accuracy and a 20 reduction in time to engage over a baseline 'wave to engage' gesture currently used on the Xbox 360. We present an experimental comparison of multi-touch and tangible user interfaces for basic interface actions. Twelve participants completed manipulation and acquisition tasks on an interactive surface in each of three conditions: tangible user interface; multi-touch; and mouse and puck. We found that interface control objects in the tangible condition were easiest to acquire and, once acquired, were easier more accurate to manipulate. Further qualitative analysis suggested that in the evaluated tasks tangibles offer greater adaptability of control and specifically highlighted a problem of exit error that can undermine fine-grained control in multi-touch interactions. We discuss the implications of these findings for interface design. Recently, multitasking on smart phones during navigation has emerged as a problematic social behavior due to its potential danger. In this paper, we propose to alleviate the situation by employing a sensor system for obstacle detection and aid the multi-tasking user so that one can safely navigate and carry out the on-going secondary task as effectively as possible. As such, we have implemented an ultrasonic sensor system interfaced into the smart phone that can constantly appraise the user of the incoming obstacles. We ran experiments to validate our approach, checking whether such a system would help user bump less into obstacles than without, and observe their multitasking behavior such as the physical attentional switch. Our experiments have shown that with the aid of the sensor system, the user's attention switch was significantly reduced, however, there was no differences in the performance (e.g. no. of collision) because the momentary strategic spatial and path planning was quite effective with slow pace and light pedestrian traffic. We conclude therefore human dual or multitasking ability is sufficient to overlap "casual" video watching and "slow" navigation. However, with more stringent task, e.g. text messaging and heavier traffic, we expect the reduced attention shift to ultimately improve the performance as well. Remote control robots are being found in an increasing number of application domains, including search and rescue, exploration, and reconnaissance. There is a large body of HRI research that investigates interface design for remote navigation, control, and sensor monitoring, while aiming for interface enhancements that benefit the remote operator such as improving ease of use, reducing operator mental load, and maximizing awareness of a robot's state and remote environment. Even though many remote control robots have multi-degree-of-freedom robotic manipulator arms for interacting with the environment, there is only limited research into easy-to-use remote control interfaces for such manipulators, and many commercial robotic products are still using simplistic interface technologies such as keypads or gamepads with arbitrary mappings to arm morphology. In this paper, we present an original interface for the remote control of a multi-degree of freedom robotic arm. We conducted a controlled experiment to compare our interface to an existing commercial keypad interface and detail our results that indicate our interface was easier to use, required less cognitive task load, and enabled people to complete tasks more quickly. In this paper, we present an original interface for the remote control of a multi-degree of freedom robotic arm. We conducted a controlled experiment to compare our interface to an existing commercial keypad interface and detail our results that indicate our interface was easier to use, required less cognitive task load, and enabled people to complete tasks more quickly. Robust remote teleoperation of high-DOF manipulators is of critical importance across a wide range of robotics applications. Contemporary robot manipulation interfaces primarily utilize a free-positioning pose specification approach to independently control each axis of translation and orientation in free space. In this work, we present two novel interfaces, constrained positioning and point-and-click, which incorporate scene information, including points-of-interest and local surface geometry, into the grasp specification process. We also present results of a user study evaluation comparing the effects of increased use of scene information in grasp pose specification algorithms for general object manipulation. The results of our study show that constrained positioning and point-and-click significantly outperform the widely used free positioning approach by significantly reducing the number of grasping errors and the number of user interactions required to specify poses. Furthermore, the point-and-click interface significantly increased the number of tasks users were able to complete. This paper examines the hypothesis that information flow, from device to user, is a vital part of skilled activity in human-computer interaction. Two studies are reported. The first study questions users of keyboard-driven word processors about the effects of cursor-movement, finding and word-deletion commands in various contexts. The second study questions users of the Apple Macintosh-based systems, MacWrite and Microsoft Word, about the behaviour of the menu-driven find command. In both studies it is discovered that users often do not know the precise effects of frequently-used actions, such as the final position of the cursor, even though these effects are vital for future planning. It is concluded that even experienced users must acquire the information they need from the device's display during interactions, and that they do not necessarily remember regular details that are available in this way. This conclusion conflicts with those current models of user psychology that assume routine skill relies on complete mental specifications of methods for performing tasks. Bare hand pointing on touch screens both benefits and suffers from the nature of direct input. This work explores techniques to overcome its limitations. Our goal is to design interaction tools allowing pixel level pointing in a fast and efficient manner. Based on several cycles of iterative design and testing, we propose two techniques: Cross-Keys that uses discrete taps on virtual keys integrated with a crosshair cursor, and an analog Precision-Handle that uses a leverage (gain) effect to amplify movement precision from the user's finger tip to the end cursor. We conducted a formal experiment with these two techniques, in addition to the previously known Zoom-Pointing and Take-Off as baseline anchors. Both subjective and performance measurements indicate that Precision-Handle and Cross-Keys complement existing techniques for touch screen interaction. Animated human characters in everyday scenarios must interact with the environment using their hands. Captured human motion can provide a database of realistic examples. However, examples involving contact are difficult to edit and retarget; realism can suffer when a grasp does not appear secure or when an apparent impact does not disturb the hand or the object. Physically based simulations can preserve plausibility through simulating interaction forces. However, such physical models must be driven by a controller, and creating effective controllers for new motion tasks remains a challenge. In this paper, we present a controller for physically based grasping that draws from motion capture data. Our controller explicitly includes passive and active components to uphold compliant yet controllable motion, and it adds compensation for movement of the arm and for gravity to make the behavior of passive and active components less dependent on the dynamics of arm motion. Given a set of motion capture grasp examples, our system solves for all but a small set of parameters for this controller automatically. We demonstrate results for tasks including grasping and two-hand interaction and show that a controller derived from a single motion capture example can be used to form grasps of different object geometries. In this paper, we address the problem of simultaneously achieving low trajectory tracking errors and compliant control without using explicit mathematical models of task dynamics. To achieve this goal, we propose a new movement representation called compliant movement primitives (CMPs), which encodes position trajectory and associated torque profiles and can be learned from a single user demonstration. With the proposed control framework, the robot can remain compliant and consequently safe for humans sharing its workspace, even if high trajectory tracking accuracy is required. We developed a statistical learning approach that can use a database of existing CMPs and compute new ones, adapted for novel task variations. The proposed approach was evaluated on a Kuka LWR-4 robot performing 1) a discrete pick-and-place task with objects of varying weight and 2) a periodic handle turning operation. The evaluation of the discrete task showed a 15-fold decrease of the tracking error while exhibiting compliant behavior compared to the standard feedback control approach. It also indicated no significant rise in the tracking error while using generalized primitives computed by the statistical learning method. With respect to unforeseen collisions, the proposed approach resulted in a 75 drop of contact forces compared to standard feedback control. The periodic task demonstrated on-line use of the proposed approach to accomplish a task of handle turning. This paper proposes a planning framework to deal with the problem of computing the motion of a robot with dual arm hand, during an object pick-and-place task. We consider the situation where the start and goal configurations of the object constrain the robot to grasp the object with one hand, to give it to the other hand, before placing it in its final configuration. To realize such a task, the proposed framework treats the grasp computation, for one or two multi-fingered hands, of an arbitrarily-shaped object, the exchange configuration and finally the motion of the robot arms and body. In order to improve the planner performance, a context-independent grasp list is computed offline for each hand and for the given object as well as computed offline roadmap that will be adapted according to the environment composition. Simulation results show the planner performance on a complex scenario. The approach of inferring user's intended task and optimizing low-level robot motions has promise for making robot teleoperation interfaces more intuitive and responsive. But most existing methods assume a finite set of candidate tasks, which limits a robot's functionality. This paper proposes the notion of freeform tasks that encode an infinite number of possible goals (e.g., desired target positions) within a finite set of types (e.g., reach, orient, pick up). It also presents two technical contributions to help make freeform UIs possible. First, an intent predictor estimates the user's desired task, and accepts freeform tasks that include both discrete types and continuous parameters. Second, a cooperative motion planner continuously updates the robot's trajectories to achieve the inferred tasks by repeatedly solving optimal control problems. The planner is designed to respond interactively to changes in the indicated task, avoid collisions in cluttered environments, handle time-varying objective functions, and achieve high-quality motions using a hybrid of numerical and sampling-based techniques. The system is applied to the problem of controlling a 6D robot manipulator using 2D mouse input in the context of two tasks: static target reaching and dynamic trajectory tracking. Simulations suggest that it enables the robot to reach intended targets faster and to track intended trajectories more closely than comparable techniques. This paper explores the problem of autonomous, in-hand regrasping-the problem of moving from an initial grasp on an object to a desired grasp using the dexterity of a robot's fingers. We propose a planner for this problem which alternates between finger gaiting, and in-grasp manipulation. Finger gaiting enables the robot to move a single finger to a new contact location on the object, while the remaining fingers stably hold the object. In-grasp manipulation moves the object to a new pose relative to the robot's palm, while maintaining the contact locations between the hand and object. Given the object's geometry (as a mesh), the hand's kinematic structure, and the initial and desired grasps, we plan a sequence of finger gaits and object reposing actions to reach the desired grasp without dropping the object. We propose an optimization based approach and report in-hand regrasping plans for 5 objects over 5 in-hand regrasp goals each. The plans generated by our planner are collision free and guarantee kinematic feasibility. Both recognizing human behavior and understanding a user's mobility from sensor data are critical issues in ubiquitous computing systems. As a kind of user behavior, the transportation modes, such as walking, driving, etc., that a user takes, can enrich the user's mobility with informative knowledge and provide pervasive computing systems with more context information. In this paper, we propose an approach based on supervised learning to infer people's motion modes from their GPS logs. The contribution of this work lies in the following two aspects. On one hand, we identify a set of sophisticated features, which are more robust to traffic condition than those other researchers ever used. On the other hand, we propose a graph-based post-processing algorithm to further improve the inference performance. This algorithm considers both the commonsense constraint of real world and typical user behavior based on location in a probabilistic manner. Using the GPS logs collected by 65 people over a period of 10 months, we evaluated our approach via a set of experiments. As a result, based on the change point-based segmentation method and Decision Tree-based inference model, the new features brought an eight percent improvement in inference accuracy over previous result, and the graph-based post-processing achieve a further four percent enhancement. Robust manipulation and insertion of small parts can be challenging because of the small tolerances typically involved. The key to robust control of these kinds of manipulation interactions is accurate tracking and control of the parts involved. Typically, this is accomplished using visual servoing or force-based control. However, these approaches have drawbacks. Instead, we propose a new approach that uses tactile sensing to accurately localize the pose of a part grasped in the robot hand. Using a feature-based matching technique in conjunction with a newly developed tactile sensing technology known as GelSight that has much higher resolution than competing methods, we synthesize high-resolution height maps of object surfaces. As a result of these high-resolution tactile maps, we are able to localize small parts held in a robot hand very accurately. We quantify localization accuracy in benchtop experiments and experimentally demonstrate the practicality of the approach in the context of a small parts insertion problem. The seemingly simple everyday actions of moving limb and body to accomplish a motor task or interact with the environment are incredibly complex. To reach for a target we first need to sense the target's position with respect to an external coordinate system; we then need to plan a limb trajectory which is executed by issuing an appropriate series of neural commands to the muscles. These, in turn, exert appropriate forces and torques on the joints leading to the desired movement of the arm. Here we review some of the earlier work as well as more recent studies on the control of human movement, focusing on behavioral and modeling studies dealing with task space and joint-space movement planning. At the task level, we describe studies investigating trajectory planning and inverse kinematics problems during point-to-point reaching movements as well as two-dimensional (2D) and three-dimensional (3D) drawing movements. We discuss models dealing with the two-thirds power law, particularly differential geometrical approaches dealing with the relation between path geometry and movement velocity. We also discuss optimization principles such as the minimum-jerk model and the isochrony principle for point-to-point and curved movements. We next deal with joint-space movement planning and generation, discussing the inverse kinematics problem and common solutions to the problems of kinematic redundancy. We address the question of which reference frames are used by the nervous system and review studies examining the employment of kinematic constraints such as Donders' and Listing's laws. We also discuss optimization approaches based on Riemannian geometry. One principle of motor coordination during human locomotion emerging from this body of work is the intersegmental law of coordination. However, the nature of the coordinate systems underlying motion planning remains of interest as they are related to the principles governing the control of human arm movements.