Abstract Interaction in virtual reality (VR) environments (e.g. grasping and manipulating virtual objects) is essential to ensure a pleasant and immersive experience. In this work, we propose a visually realistic, flexible and robust grasping system that enables real-time interactions in virtual environments. Resulting grasps are visually realistic because hand is automatically fitted to the object shape from a position and orientation determined by the user using the VR handheld controllers (e.g. Oculus Touch motion controllers). Our approach is flexible because it can be adapted to different hand meshes (e.g. human or robotic hands) and it is also easily customizable. Moreover, it enables interaction with different objects regardless their geometries. In order to validate our proposal, an exhaustive qualitative and quantitative performance analysis has been carried out. On one hand, qualitative evaluation was used in the assessment of abstract aspects, such as motor control, finger movement realism, and interaction realism. On the other hand, for the quantitative evaluation a novel metric has been proposed to visually analyze the performed grips. Performance analysis results indicate that previous experience with our grasping system is not a prerequisite for an enjoyable, natural and intuitive VR interaction experience.
The selection process is also constrained by the hand high degree of freedom (DOF). In order to deal with dimensionality and redundancy many researchers have used techniques such as principal component analysis (PCA) @cite_1 @cite_28 . For the same purpose, @cite_22 studied the correlations between hand DOFs aiming to simplify hand models reducing DOF number. The results suggest to simplify hand models by reducing DOFs from 50 to 15 for both hands in conjunction without loosing relevant features.
@cite_28 @cite_1 @cite_22
2138983671 2066864006
Passive sensing of human hand and limb motion is important for a wide range of applications from human-computer interaction to athletic performance measurement. High degree of freedom articulated mechanisms like the human hand are difficult to track because of their large state space and complex image appearance. This article describes a model-based hand tracking system, called DigitEyes, that can recover the state of a 27 DOF hand model from ordinary gray scale images at speeds of up to 10 Hz. We propose a method that relies on markerless visual observations to track the full articulation of two hands that interact with each-other in a complex, unconstrained manner. We formulate this as an optimization problem whose 54-dimensional parameter space represents all possible configurations of two hands, each represented as a kinematic structure with 26 Degrees of Freedom (DoFs). To solve this problem, we employ Particle Swarm Optimization (PSO), an evolutionary, stochastic optimization method with the objective of finding the two-hands configuration that best explains observations provided by an RGB-D sensor. To the best of our knowledge, the proposed method is the first to attempt and achieve the articulated motion tracking of two strongly interacting hands. Extensive quantitative and qualitative experiments with simulated and real world image sequences demonstrate that an accurate and efficient solution of this problem is indeed feasible. Due to occlusions, the estimation of the full pose of a human hand interacting with an object is much more challenging than pose recovery of a hand observed in isolation. In this work we formulate an optimization problem whose solution is the 26-DOF hand pose together with the pose and model parameters of the manipulated object. Optimization seeks for the joint hand-object model that (a) best explains the incompleteness of observations resulting from occlusions due to hand-object interaction and (b) is physically plausible in the sense that the hand does not share the same physical space with the object. The proposed method is the first that solves efficiently the continuous, full-DOF, joint hand-object tracking problem based solely on markerless multicamera input. Additionally, it is the first to demonstrate how hand-object interaction can be exploited as a context that facilitates hand pose estimation, instead of being considered as a complicating factor. Extensive quantitative and qualitative experiments with simulated and real world image sequences as well as a comparative evaluation with a state-of-the-art method for pose estimation of isolated hands, support the above findings. Subjects were asked to shape the right hand as if to grasp and use a large number of familiar objects. The chosen objects typically are held with a variety of grips, including “precision” and “power” grips. Static hand posture was measured by recording the angular position of 15 joint angles of the fingers and of the thumb. Although subjects adopted distinct hand shapes for the various objects, the joint angles of the digits did not vary independently. Principal components analysis showed that the first two components could account for .80 of the variance, implying a substantial reduction from the 15 degrees of freedom that were recorded. However, even though they were small, higher-order (more than three) principal components did not represent random variability but instead provided additional information about the object. These results suggest that the control of hand posture involves a few postural synergies, regulating the general shape of the hand, coupled with a finer control mechanism providing for small, subtle adjustments. Because the postural synergies did not coincide with grip taxonomies, the results suggest that hand posture may be regulated independently from the control of the contact forces that are used to grasp an object. A closed-form inverse kinematics solver for non-redundant reconfigurable robots is developed based on the product-of-exponentials (POE) formula. Its novelty lies in the use of POE reduction techniques and subproblems to obtain inverse kinematics solutions of a large number of possible configurations in a systematic and convenient way. Three reduction techniques are introduced to simplify the POE equations. Eleven types of subproblems containing geometric solutions of those simplified equations are identified and solved. Based on the sequence and types of robot joints, the solved sub-problems can be re-used for inverse kinematics of different robot configurations. This solver can cope with closed-form inverse kinematics of all robots with DOFs of 4 or less, 90 percent of the 5-DOF robots and 50 percent of the 6-DOF robots, as well as frequently used industrial robots with both prismatic and revolute joints. The solver is implemented as a C++ software package and is demonstrated through an example. Developmental robotics suggests that the forward and inverse kinematics should be learned through a sensory-motor mapping, instead of being programmed in advance. Motor babbling and goal babbling are two common approaches to generate training samples used to acquire a sensory-motor mapping. Motor babbling typically needs a considerable amount of training data and time to acquire a sufficient mapping, while goal babbling poses difficulties on how to select appropriate goals. In this paper, we propose a neurobiologically-inspired system to progressively learn a sensory-motor mapping bootstrapped from a simple constrained DOF exploration, which generates much less training data than motor babbling. Our proposed system is designed according to two neurobiologically-inspired paradigms: spatiotemporal prediction and uniformity. The spatiotemporal prediction capability facilitates the acquisition of sensory-motor mappings with less amount of training data on the one hand, and facilitates robust behaviour on the other hand. The uniform system design structure is the foundation for building a scalable architecture for cognitive development. We use an improved version of our predictive action selector (PAS) as building block of our system. We validate a PAS on a 2 DOF robot head where the robot learns object tracking and evading. Then we validate a second PAS on a 5 DOF arm where it learns reaching. This paper describes geometric and kinematic modelling steps of a class of an hybrid hyper-redundant robot (HHRR). The main problem of such complex manipulator robot structures is the synthesis of an appropriate control of n number of degrees of freedom (DOF). Thus, it is important at the design step, to develop geometric and kinematic models allowing to reconstruct the kinematical behavior of n concatenated modules. The studied structure is considered as modular with multiple DOF and is built by connecting in serial many parallel modules. The coordinate frame distribution is based on space curve differential geometry. The inverse Geometric and kinematic models are calculated analytically for a n-DOF of HHRR manipulator while the final application is applied to the case of a 3-modules robot (18-DOF). A 30 degree-of-freedom (DOF) planar hyper-redundant manipulator is constructed in order to validate and demonstrate the authors' previous work on a framework for hyper-redundant robot kinematic analysis and algorithm development. The implementation of this robot is reviewed, and experimental work in hyper-redundant manipulation, obstacle avoidance, and locomotion is summarized. Potential benefits of high-DOF manipulators include superior obstacle avoidance ability, increased robustness with respect to mechanical failure, and the ability to perform new forms of robot locomotion and grasping. > Abstract This article reports an experimental study that aimed to quantitatively analyze motion coordination patterns across digits 2–5 (index to little finger), and examine the kinematic synergies during manipulative and gestic acts. Twenty-eight subjects (14 males and 14 females) performed two types of tasks, both right-handed: (1) cylinder-grasping that involved concurrent voluntary flexion of digits 2–5, and (2) voluntary flexion of individual fingers from digit 2 to 5 (i.e., one at a time). A five-camera opto-electronic motion capture system measured trajectories of 21 miniature reflective markers strategically placed on the dorsal surface landmarks of the hand. Joint angular profiles for 12 involved flexion–extension degrees of freedom (DOF's) were derived from the measured coordinates of surface markers. Principal components analysis (PCA) was used to examine the temporal covariation between joint angles. A mathematical modeling procedure, based on hyperbolic tangent functions, characterized the sigmoidal shaped angular profiles with four kinematically meaningful parameters. The PCA results showed that for all the movement trials ( n =280), two principal components accounted for at least 98 of the variance. The angular profiles ( n =2464) were accurately characterized, with the mean (±SD) coefficient of determination ( R 2 ) and root-mean-square-error (RMSE) being 0.95 (±0.12) and 1.03° (±0.82°), respectively. The resulting parameters which quantified both the spatial and temporal aspects of angular profiles revealed stereotypical patterns including a predominant (87 of all trials) proximal-to-distal flexion sequence and characteristic interdependence – involuntary joint flexion induced by the voluntarily flexed joint. The principal components' weights and the kinematic parameters also exhibited qualitatively similar variation patterns. Motor control interpretations and new insights regarding the underlying synergistic mechanisms, particularly in relation to previous findings on force synergies, are discussed. Dexterous multi-fingered hands are extremely versatile and provide a generic way to perform multiple tasks in human-centric environments. However, effectively controlling them remains challenging due to their high dimensionality and large number of potential contacts. Deep reinforcement learning (DRL) provides a model-agnostic approach to control complex dynamical systems, but has not been shown to scale to high-dimensional dexterous manipulation. Furthermore, deployment of DRL on physical systems remains challenging due to sample inefficiency. Thus, the success of DRL in robotics has thus far been limited to simpler manipulators and tasks. In this work, we show that model-free DRL with natural policy gradients can effectively scale up to complex manipulation tasks with a high-dimensional 24-DoF hand, and solve them from scratch in simulated experiments. Furthermore, with the use of a small number of human demonstrations, the sample complexity can be significantly reduced, and enable learning within the equivalent of a few hours of robot experience. We demonstrate successful policies for multiple complex tasks: object relocation, in-hand manipulation, tool use, and door opening. We consider the K user Multiple Input Multiple Output (MIMO) Gaussian interference channel with M antennas at each transmitter and N antennas at each receiver. It is assumed that channel coefficients are fixed and are available at all transmitters and at all receivers. The main objective of this paper is to characterize the total Degrees Of Freedom (DOF) for this channel. Using a new interference alignment technique which has been recently introduced in [1], we show that MN over M+N K degrees of freedom can be achieved for almost all channel realizations. Also, a new upper-bound on the total DOF for this channel is derived. This upper-bound coincides with our achievable DOF for K ≥ K u ≜ M+N over gcd(M,N) where gcd(M,N) denotes the greatest common divisor of M and N. This gives an exact characterization of DOF for MIMO Gaussian interference channel in the case of K ≥ K u . We present a technique to estimate the egomotion of an RGB-D sensor based on rotations of functions defined on the unit sphere. In contrast to traditional approaches, our technique is not based on image features and does not require correspondences to be generated between frames of data. Instead, consecutive functions are correlated using spherical harmonic analysis. An Extended Gaussian Image (EGI), created from the local normal estimates of a point cloud, defines each function. Correlations are efficiently computed using Fourier transformations, resulting in a 3 Degree of Freedom (3-DoF) rotation estimate. An Iterative Closest Point (ICP) process then refines the initial rotation estimate and adds a translational component, yielding a full 6-DoF egomotion estimate. The focus of this work is to investigate the merits of using spherical harmonic analysis for egomotion estimation by comparison with alternative 6-DoF methods. We compare the performance of the proposed technique with that of stand-alone ICP and image feature based methods. As with other egomotion techniques, estimation errors accumulate and degrade results, necessitating correction mechanisms for robust localization. For this report, however, we use the raw estimates; no filtering or smoothing processes are applied. In-house and external benchmark data sets are analyzed for both runtime and accuracy. Results show that the algorithm is competitive in terms of both accuracy and runtime, and future work will aim to combine the various techniques into a more robust egomotion estimation framework. We consider a single-hop interference network with K transmitters and J receivers, all having M antennas. Each transmitter emits an independent message and each receiver requests an arbitrary subset of the messages. This generalizes the well-known K -user M-antenna interference channel, where each message is requested by a unique receiver. For our setup, we derive the degrees of freedom (DoF) region. The achievability scheme generalizes the interference alignment schemes proposed by Cadambe and Jafar. In particular, we achieve general points in the DoF region by using multiple base vectors and aligning all interferers at a given receiver to the interferer with the largest DoF. As a byproduct, we obtain the DoF region for the original interference channel. We also discuss extensions of our approach where the same region can be achieved by considering a reduced set of interference alignment constraints, thus reducing the time-expansion duration needed. The DoF region for the considered system depends only on a subset of receivers whose demands meet certain characteristics. The geometric shape of the DoF region is also discussed. We present a novel optimization-based motion planning algorithm for high degree-of-freedom (DOF) robots in dynamic environments. Our approach decomposes the high-dimensional motion planning problem into a sequence of low-dimensional sub-problems. We compute collision-free and smooth paths using optimization-based planning and trajectory perturbation for each sub-problem. The overall algorithm does not require a priori knowledge about global motion or trajectories of dynamic obstacles. Rather, we compute a conservative local bound on the position or trajectory of each obstacle over a short time and use the bound to incrementally compute a collision-free trajectory for the robot. The high-DOF robot is treated as a tightly coupled system, and we incrementally use constrained coordination to plan its motion. We highlight the performance of our planner in simulated environments on robots with tens of DOFs. In this letter, the degrees of freedom (DoF) outer bound is characterized for an L-cell constant cellular interfering network by using linear interference alignment (IA). For a symmetric network, it means that no matter how many cells and users per cell there are, the total achievable DoF is bounded by M+N, where M and N denote the number of antennas equipped on users and base stations. Besides, a DoF achievable algorithm is also proposed. Although the DoF outer bound cannot be reached, more DoF can be obtained compared with orthogonal schemes. This paper deals with the experimental identification of the dynamic parameters of parallel machines. The dynamic parameters are estimated by using the weighted least squares solution of an over determined linear system obtained from the sampling of the dynamic model along a closed loop exciting trajectory. Experimental results are exhibited for the H4 robot, a fully parallel structure providing 3 degrees of freedom (DOF) in translation and 1 DOF in rotation. A comparative study is performed depending on the available measurements, i.e., different sensor locations (motor, end effector). We present an approach to control a 6-degree-of-freedom (DOF) manipulator using an uncalibrated visual servoing (VS) approach that addresses the challenges of choosing proper image features for target objects and designing a VS controller to enhance the tracking performance. The main contribution of this paper is the definition of a new virtual visual space (image space). A novel stereo camera model employing virtual orthogonal cameras is used to map 6-D poses from Cartesian space to this virtual visual space . Each component of the 6-D pose vector defined in this virtual visual space is linearly independent, leading to a full-rank @math image Jacobian matrix , which allows avoiding classical problems, such as image space singularities and local minima. Furthermore, the control for rotational and translational motion of robot is decoupled due to the diagonal image Jacobian. Finally, simulation results with an eye-to-hand robotic system confirm the improvement in controller stability and motion performance with respect to conventional VS approaches. Experimental results on a 6-DOF industrial robot are provided to illustrate the effectiveness of the proposed method and the feasibility of using this method in practical scenarios. Animating hand-object interactions is a frequent task in applications such as the production of 3d movies. Unfortunately this task is difficult due to the hand's many degrees of freedom and the constraints on the hand motion imposed by the geometry of the object. However, the causality between the object state and the hand's pose can be exploited in order to simplify the animation process. In this paper, we present a method that takes an animation of an object as input and automatically generates the corresponding hand motion. This approach is based on the simple observation that objects are easier to animate than hands, since they usually have fewer degrees of freedom. The method is data-driven; sequences of hands manipulating an object are captured semi-automatically with a structured-light setup. The training data is then combined with a new animation of the object in order to generate a plausible animation featuring the hand-object interaction. Consider a K-user flat fading MIMO interference channel where the kth transmitter (or receiver) is equipped with Mk (respectively Nk) antennas. If an exponential (in K) number of generic channel extensions are used either across time or frequency, Cadambe and Jafar [1] showed that the total achievable degrees of freedom (DoF) can be maximized via interference alignment, resulting in a total DoF that grows linearly with A even if Mk and Nk are bounded. In this work we consider the case where no channel extension is allowed, and establish a general condition that must be satisfied by any degrees of freedom tuple (d1. d2....dK) achievable through linear interference alignment. For a symmetric system with Mk = M, Nk = N, dk = d for all k, this condition implies that the total achievable DoF cannot grow linearly with K, and is in fact no more than K(M + N) (K + 1). We also show that this bound is tight when the number of antennas at each transceiver is divisible by d, the number of data streams per user. We present a novel 6-DoF cable driven manipulator for handheld robotic tasks. Based on a coupled tendon approach, the arm is optimized to maximize movement speed and configuration space while reducing the total mass of the arm. We propose a space carving approach to design optimal link geometry maximizing structural strength and joint limits while minimizing link mass. The design improves on similar non-handheld tendon-driven manipulators and reduces the required number of actuators to one per DoF. As the manipulator has one redundant joint, we present a 5-DoF inverse kinematics solution for the end effector pose. The inverse kinematics is solved by splitting the 6-DoF problem into two coupled 3-DoF problems and merging their results. A method for gracefully degrading the output of the inverse kinematics is described for cases where the desired end effector pose is outside the configuration space. This is useful for settings where the user is in the control loop and can help the robot to get closer to the desired location. The design of the handheld robot is offered as open source. While our results and tools are aimed at handheld robotics, the design and approach is useful to non-handheld applications.