RL is commonly used in the task of image captioning to reduce the train-test discrepancy [1]}. Since we formulate the image captioning as the dependent multi-task problem, the inter-task train-test discrepancy also needs to be tackled. During the MLE optimization period, Gumbel-softmax approximation is used but will improve the gradient variance because of the temperature parameter [2]}. Therefore, we use a multi-agent reinforcement learning strategy to train the model end-to-end based on the SCRL strategy on the two tasks alternatively.
