Visual Radial Basis Q-Network
While reinforcement learning (RL) from raw images has been largely
investigated in the last decade, existing approaches still suffer from a number
of constraints. The high input dimension is often handled using either expert
knowledge to extract handcrafted features or environment encoding through
convolutional networks. Both solutions require numerous parameters to be
optimized. In contrast, we propose a generic method to extract sparse features
from raw images with few trainable parameters. We achieved this using a Radial
Basis Function Network (RBFN) directly on raw image. We evaluate the
performance of the proposed approach for visual extraction in Q-learning tasks
in the Vizdoom environment. Then, we compare our results with two Deep
Q-Network, one trained directly on images and another one trained on feature
extracted by a pretrained auto-encoder. We show that the proposed approach
provides similar or, in some cases, even better performances with fewer
trainable parameters while being conceptually simpler.