where the difference from the vanilla Transformer is that \(\text{Transformer}_{(s)}(\cdot )\)  does not contain the position embedding. Then we encode the generated captions \(\widetilde{y}^i\)  through BERT [1]}:
\(H_s^i = \text{BERT}(\widetilde{y}^i)\) 
\(h_s^i = \text{pool}(H_s^i)\) 
