Balancing New Against Old Information: The Role of Surprise
Surprise is a widely used concept describing a range of phenomena from unexpected events to behavioral responses. We propose a measure of surprise, to arrive at a new framework for surprise driven learning. There are two components to this framework: (i) a confidence adjusted surprise measure to capture environmental statistics as well as subjective beliefs, (ii) a surprise minimization learning rule, or SMiLe rule, which dynamically adjusts the balance between new and old information without making prior assumptions about the temporal statistics of the environment. We apply our framework to a dynamic decision making task and a maze exploration task to demonstrate that it is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes. Our proposed surprise modulated belief update algorithm provides a framework to study the behavior of humans and animals encountering surprising events.
ArXiv