Prototyping three key properties of specific curiosity in computational reinforcement learning
Curiosity for machine agents has been a focus of intense research. The study
of human and animal curiosity, particularly specific curiosity, has unearthed
several properties that would offer important benefits for machine learners,
but that have not yet been well-explored in machine intelligence. In this work,
we introduce three of the most immediate of these properties -- directedness,
cessation when satisfied, and voluntary exposure -- and show how they may be
implemented together in a proof-of-concept reinforcement learning agent;
further, we demonstrate how the properties manifest in the behaviour of this
agent in a simple non-episodic grid-world environment that includes
curiosity-inducing locations and induced targets of curiosity. As we would
hope, the agent exhibits short-term directed behaviour while updating long-term
preferences to adaptively seek out curiosity-inducing situations. This work
therefore presents a novel view into how specific curiosity operates and in the
future might be integrated into the behaviour of goal-seeking, decision-making
agents in complex environments.