Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies
Autonomous open-ended learning is a relevant approach in machine learning and
robotics, allowing the design of artificial agents able to acquire goals and
motor skills without the necessity of user assigned tasks. A crucial issue for
this approach is to develop strategies to ensure that agents can maximise their
competence on as many tasks as possible in the shortest possible time.
Intrinsic motivations have proven to generate a task-agnostic signal to
properly allocate the training time amongst goals. While the majority of works
in the field of intrinsically motivated open-ended learning focus on scenarios
where goals are independent from each other, only few of them studied the
autonomous acquisition of interdependent tasks, and even fewer tackled
scenarios where goals involve non-stationary interdependencies. Building on
previous works, we tackle these crucial issues at the level of decision making
(i.e., building strategies to properly select between goals), and we propose a
hierarchical architecture that treating sub-tasks selection as a Markov
Decision Process is able to properly learn interdependent skills on the basis
of intrinsically generated motivations. In particular, we first deepen the
analysis of a previous system, showing the importance of incorporating
information about the relationships between tasks at a higher level of the
architecture (that of goal selection). Then we introduce H-GRAIL, a new system
that extends the previous one by adding a new learning layer to store the
autonomously acquired sequences of tasks to be able to modify them in case the
interdependencies are non-stationary. All systems are tested in a real robotic
scenario, with a Baxter robot performing multiple interdependent reaching
tasks.