Interaction-Grounded Learning with Action-inclusive Feedback
Consider the problem setting of Interaction-Grounded Learning (IGL), in which
a learner's goal is to optimally interact with the environment with no explicit
reward to ground its policies. The agent observes a context vector, takes an
action, and receives a feedback vector, using this information to effectively
optimize a policy with respect to a latent reward function. Prior analyzed
approaches fail when the feedback vector contains the action, which
significantly limits IGL's success in many potential scenarios such as
Brain-computer interface (BCI) or Human-computer interface (HCI) applications.
We address this by creating an algorithm and analysis which allows IGL to work
even when the feedback vector contains the action, encoded in any fashion. We
provide theoretical guarantees and large-scale experiments based on supervised
datasets to demonstrate the effectiveness of the new approach.