Incorporating Future Information  
Most recently, [1]}, [2]}, [3]}, [4]} and [5]} attempted to exploit the future information to boost the performance for sequence-to-sequence learning systems.
However, their modelings are different from ours.
Specifically, to exploit the future information, at each time step, [1]} first adopt the fine-tuned BERT [7]} to encode the words that will be generated in the future to acquire the future information.
And then the obtained future information is exploited as an extra supervision to guide the current word generation based on the previous generated words in conventional sequence-to-sequence models.
In this way, [1]} can incorporate the future information for predicting the present to improve the performance of text generation.
For [2]}, [3]} and [4]}, given a time stamp in sequence generation, in addition to current target, they further predict the future words, i.e., [2]}, [3]} further predict target word one more step ahead, and [4]} predicts the rest of the sequence. The prediction accuracy of future targets and current target are combined together in parameter optimization.
While in our proposed approach, we use the hidden states of future time stamps to explicitly guide the attention calculation of current one. In other words, we aim to better ground current target word to proper image regions which alleviates the attention deviation problem, resulting in improved performance of both grounding and captioning.
One of them [5]} also conduct a similar effort. The cyclical training regimen proposed in [5]} is similar to the idea of our Constant Prophet Attention.
In particular, [5]} only consider the current target to model the future information, while we leverage the bidirectional knowledge to enhance the modeling of the future information.
Besides, we further propose the Dynamic Prophet Attention and the Prophet Knowledge Distillation trick for better exploiting the future information.
