Since we use RL optimization to learn cluster centroids, we compare our full model (CLASTER) to a baseline where the centroids are updated using standard SGD instead of RL (referred to in Table REF  as “CLASTER w/o RL"). As a comparison, we also replace our centroid-learning with NetVLAD [1]}. The input to NetVLAD are also the joint visual-semantic features. We see that our model outperforms NetVLAD by an average of 4.7% and the CLASTER w/o RL by 7.3% on the UCF101 dataset. One reason for this difference is that the loss is
back-propagated through multiple parts of the model before
reaching the centroids. However, with RL the centroids are
directly updated using the reward signal.
We see that the cluster optimization with RL gives a clear performance improvement. Section REF  explores how the clusters change after this optimization. In a nutshell, the RL optimization essentially makes the clusters cleaner moving most instances in a class to the same cluster.
<TABLE>