In imitation learning, behavior learning is generally done using the features extracted from the demonstration data. Recent deep learning algorithms enable the development of machine learning methods that can get high dimensional data as an input. In this work, we use imitation learning to teach the robot to dribble the ball to the goal. We use B-Human robot software to collect demonstration data and a deep convolutional network to represent the policies. We use top and bottom camera images of the robot as input and speed commands as outputs. The CNN policy learns the mapping between the series of images and speed commands. In 3D realistic robotics simulator experiments, we show that the robot is able to learn to search the ball and dribble the ball, but it struggles to align to the goal. The best-proposed policy model learns to score 4 goals out of 20 test episodes.
Imitation learning is a promising research area since it has the potential of enabling non-technical domain experts to teach their expertise to a computational machine. There are many ways of demonstrating a behavior to a robot such as teleoperation and exoskeleton suit. In this study, a high-level behavior control software controls the robot in simulation, and raw sensor readings and mid-level actions are collected as demonstration data set. Our mid-level actions are 2D speed commands. We use teleoperation based data demonstration and direct state-action policy representation according to the categorization of Husseion , and Argall since we learn state-action mapping @cite_8 @cite_18 .
@cite_18 @cite_8
2735089625 1989784841 2545129501 2032740574
Deep generative models have recently shown great promise in imitation learning for motor control. Given enough data, even supervised approaches can do one-shot imitation learning; however, they are vulnerable to cascading failures when the agent trajectory diverges from the demonstrations. Compared to purely supervised methods, Generative Adversarial Imitation Learning (GAIL) can learn more robust controllers from fewer demonstrations, but is inherently mode-seeking and more difficult to train. In this paper, we show how to combine the favourable aspects of these two approaches. The base of our model is a new type of variational autoencoder on demonstration trajectories that learns semantic policy embeddings. We show that these embeddings can be learned on a 9 DoF Jaco robot arm in reaching tasks, and then smoothly interpolated with a resulting smooth interpolation of reaching behavior. Leveraging these policy representations, we develop a new version of GAIL that (1) is much more robust than the purely-supervised controller, especially with few demonstrations, and (2) avoids mode collapse, capturing many diverse behaviors when GAIL on its own does not. We demonstrate our approach on learning diverse gaits from demonstration on a 2D biped and a 62 DoF 3D humanoid in the MuJoCo physics environment. Learning by imitation renders a means for more natural human-robot interaction and is potentially the primary form of teaching. Imitation provides a potential means of automatically programming complex systems without extensive trials. This paper presents a method of imitation learning based on mapping the demonstrator's motion patterns to the observer self-posture. The self-posture is acquired through observing and recognizing the mirror image of its own body posture while performing the action in front of a mirror. First, different kinds of behavioural actions are learned through motor babbling by randomly performing different actions. The actions are learned through a novel probabilistic model called Topological Gaussian Adaptive Resonance Hidden Markov Model. During learning, the observer also extracts visual features through optical flow from self-observation in a mirror image. Second, after learning, a visuo-motor association is developed through novel Topological Gaussian Adaptive Resonance Associative Memory. Finally, after learning the demonstrator performs a similar action in front of the robot and the he robot recalls the corresponding motor command from the memory. Programming by demonstration, also called "imitation learning," offers the possibility of flexible, easily modifiable robotic systems. Full-fledged robotic imitation learning comprises many difficult subtasks. However, we argue that, at its core, imitation learning reduces to a regression problem. We propose a two-step framework in which an imitating agent first performs a regression from a high-dimensional observation space to a low-dimensional latent variable space. In the second step, the agent performs a regression from the latent variable space to a high-dimensional space representing degrees of freedom of its motor system. We demonstrate the validity of the approach by learning to map motion capture data from human actors to a humanoid robot. We also contrast use of several low-dimensional latent variable spaces, each covering a subset of agents' degrees of freedom, with use of a single, higher-dimensional latent variable space. Our findings suggest that compositing several regression models together yields qualitatively better imitation results than using a single, more complex regression model Although imitation learning is a powerful tech- nique for robot learning and knowledge acquisition from na¨ human users, it often suffers from the need for expensive human demonstrations. In some cases the robot has an insufficient number of useful demonstrations, while in others its learning ability is limited by the number of users it directly interacts with. We propose an approach that overcomes these short- comings by using crowdsourcing to collect a wider variety of examples from a large pool of human demonstrators online. We present a new goal-based imitation learning framework which utilizes crowdsourcing as a major source of human demonstration data. We demonstrate the effectiveness of our approach experimentally on a scenario where the robot learns to build 2D object models on a table from basic building blocks using knowledge gained from locals and online crowd workers. In addition, we show how the robot can use this knowledge to support human-robot collaboration tasks such as goal inference through object-part classification and missing-part prediction. We report results from a user study involving fourteen local demonstrators and hundreds of crowd workers on 16 different model building tasks.