@inproceedings{
metternich2023using,
title={Using Proto-Value Functions for Curriculum Generation in Goal-Conditioned {RL}},
author={Henrik Metternich and Ahmed Hendawy and Pascal Klink and Jan Peters and Carlo D'Eramo},
booktitle={NeurIPS 2023 Workshop on Goal-Conditioned Reinforcement Learning},
year={2023},
url={https://openreview.net/forum?id=DYd2q5wufn}
}
In this paper, we investigate the use of Proto Value Functions (PVFs) for measuring the similarity between tasks in the context of Curriculum Learning (CL). PVFs serve as a mathematical framework for generating basis functions for the state space of a Markov Decision Process (MDP).  They capture the structure of the state space manifold and have been shown to be useful for value function approximation in Reinforcement Learning (RL).  We show that even a few PVFs allow us to estimate the similarity between tasks. Based on this observation, we introduce a new algorithm called Curriculum Representation Policy Iteration (CRPI) that uses PVFs for CL, and we provide a proof of concept in a Goal-Conditioned Reinforcement Learning (GCRL) setting.
Using Proto-Value Functions for Curriculum Generation in Goal-Conditioned RL
Henrik Metternich Ahmed Hendawy Pascal Klink Jan Peters Carlo D'Eramo