Abstract Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself are crucial. In this work, we propose a novel framework for probabilistic online motion planning with online adaptation based on a bio-inspired stochastic recurrent neural network. By using learning signals which mimic the intrinsic motivation signal cognitive dissonance in addition with a mental replay strategy to intensify experiences, the stochastic recurrent network can learn from few physical interactions and adapts to novel environments in seconds. We evaluate our online planning and adaptation framework on an anthropomorphic KUKA LWR arm. The rapid online adaptation is shown by learning unknown workspace constraints sample-efficiently from few physical interactions while following given way points.
A different approach based on competence improvement with hierarchical reinforcement learning is discussed in @cite_58 . The agent is given a set of skills, or options as in the options framework, and needs to choose which skill to improve. The used motivation signal competence is implemented as the expected return of a skill to achieve a certain goal. Rewards are generated based on this competence progress and the approach is evaluated in a gridworld domain.
@cite_58
1992416110
One important role of an agent's motivational system is to choose, at any given moment, which of a number of skills the agent should attempt to improve. Many researchers have suggested “intrinsically motivated” systems that receive internal reward for model learning progress, but for the most part this notion has not been applied with respect to skill competence, or to choose between skills. In this paper we propose an agent motivated to gain competence in its environment by learning a number of skills, addressing head-on the mechanism of competence progress motivation for the purpose of governing the efficient learning of skills. We demonstrate this new approach in a simple illustrative domain and show that it outperforms a naive agent, achieving higher competence faster by focusing attention and learning effort on skills for which progress can be made while ignoring those skills that are already learned or are at the moment too difficult.