Hyperparameters. We use the pre-trained Faster R-CNN model [1]} to represent each image as an adaptive sequence of feature vectors \(R\)  where the number of detected objects \(n \in [10,100]\) . The number of encoder layers is 4 and we use the output of the second layer to predict BOC. The dimension of each feature vector \(v_{i}\)  is 2,048 and then projected to 1,024. The dimensions of the word embedding layer, attention layers, and the LSTM-based decoder are all set to 1,024. The learning rate is initialized to \(0.0001\)  and decreased by half when the CIDEr-D score does not increase in 2 epochs, with the minimum learning rate set to \(5\cdot 10^{-6}\) . The batch size is set to 50. The model is firstly optimized with MLE for 30 epochs (Gumbel sampling \(\widetilde{m}\)  after 15 epochs), and then optimized with MARL for another 35 epochs. The special setting of our model is that we can sample different \(z_c\)  to generate multiple captions. Therefore, we train an image-text selector to select the best of 20 generated captions for each image (details in Appendix ).
<TABLE><TABLE>