Thirdly, to incorporate fine-grained acoustic context [1]} we additionally concatenate to the decoder inputs vocoder features (MFCCs, \(f_0\) , periodicity) where frames corresponding to the replaced words in the original speech are masked out with zeros [2]}. Word to frame alignments are obtained using a pretrained alignment model. Additionally we concatenate to the encoder inputs a two-value 32-dim mask token to further emphasise to the model which frames correspond to the masked region.
