Grasping is a complex process involving knowledge of the object, the surroundings, and of oneself. While humans are able to integrate and process all of the sensory information required for performing this task, equipping machines with this capability is an extremely challenging endeavor. In this paper, we investigate how deep learning techniques can allow us to translate high-level concepts such as motor imagery to the problem of robotic grasp synthesis. We explore a paradigm based on generative models for learning integrated object-action representations, and demonstrate its capacity for capturing and generating multimodal, multi-finger grasp configurations on a simulated grasping dataset.
Motor imagery (MI) and motor execution (ME) are two different forms of motor representation. While motor execution is an external representation (physical performance of an action), motor imagery is the use of an internal representation for mentally an action @cite_22 @cite_1 .
@cite_1 @cite_22
2051652848 2165108033 2022965779 2145546283
Rehabilitation, for a large part may be seen as a learning process where old skills have to be re-acquired and new ones have to be learned on the basis of practice. Active exercising creates a flow of sensory (afferent) information. It is known that motor recovery and motor learning have many aspects in common. Both are largely based on response-produced sensory information. In the present article it is asked whether active physical exercise is always necessary for creating this sensory flow. Numerous studies have indicated that motor imagery may result in the same plastic changes in the motor system as actual physical practice. Motor imagery is the mental execution of a movement without any overt movement or without any peripheral (muscle) activation. It has been shown that motor imagery leads to the activation of the same brain areas as actual movement. The present article discusses the role that motor imagery may play in neurological rehabilitation. Furthermore, it will be discussed to what extent the observation of a movement performed by another subject may play a similar role in learning. It is concluded that, although the clinical evidence is still meager, the use of motor imagery in neurological rehabilitation may be defended on theoretical grounds and on the basis of the results of experimental studies with healthy subjects. We propose a general architecture for action (mimicking) and program (gesture) level visual imitation. Action-level imitation involves two modules. The viewpoint transformation (VPT) performs a "rotation" to align the demonstrator's body to that of the learner. The visuo-motor map (VMM) maps this visual information to motor data. For program-level (gesture) imitation, there is an additional module that allows the system to recognize and generate its own interpretation of observed gestures to produce similar gestures goals at a later stage. Besides the holistic approach to the problem, our approach differs from traditional work in i) the use of motor information for gesture recognition; ii) usage of context (e.g., object affordances) to focus the attention of the recognition system and reduce ambiguities, and iii) use iconic image representations for the hand, as opposed to fitting kinematic models to the video sequence. This approach is motivated by the finding of visuomotor neurons in the F5 area of the macaque brain that suggest that gesture recognition imitation is performed in motor terms (mirror) and rely on the use of object affordances (canonical) to handle ambiguous actions. Our results show that this approach can outperform more conventional (e.g., pure visual) methods. Abstract Paradigms drawn from cognitive psychology have provided new insight into covert stages of action. These states include not only intending actions that will eventually be executed, but also imagining actions, recognizing tools, learning by observation, or even understanding the behavior of other people. Studies using techniques for mapping brain activity, probing cortical excitability, or measuring the activity of peripheral effectors in normal human subjects and in patients all provide evidence of a subliminal activation of the motor system during these cognitive states. The hypothesis that the motor system is part of a simulation network that is activated under a variety of conditions in relation to action, either self-intended or observed from other individuals, will be developed. The function of this process of simulation would be not only to shape the motor system in anticipation to execution, but also to provide the self with information on the feasibility and the meaning of potential actions. In this paper, we present a novel approach for human action recognition with histograms of 3D joint locations (HOJ3D) as a compact representation of postures. We extract the 3D skelet al joint locations from Kinect depth maps using 's method [6]. The HOJ3D computed from the action depth sequences are reprojected using LDA and then clustered into k posture visual words, which represent the prototypical poses of actions. The temporal evolutions of those visual words are modeled by discrete hidden Markov models (HMMs). In addition, due to the design of our spherical coordinate system and the robust 3D skeleton estimation from Kinect, our method demonstrates significant view invariance on our 3D action dataset. Our dataset is composed of 200 3D sequences of 10 indoor activities performed by 10 individuals in varied views. Our method is real-time and achieves superior results on the challenging 3D action dataset. We also tested our algorithm on the MSR Action 3D dataset and our algorithm outperforms [25] on most of the cases.