We formulate our MEP task using InfoNCE [1]}, which is the softmax version of the noise contrastive estimation (NCE) [2]}. Let \(\tilde{\mathbf {o}}_{t}\)  be the \(t\) -th output of any of the three Transformers obtained by masking the \(t\) -th input \(\mathbf {x}_t\) . Our InfoNCE loss is then defined as
\(\mathcal {L}_{\textrm {NCE}}(\mathbf {x}, \tilde{\mathbf {o}}) = -\mathbb {E}_{\mathbf {x}} \left[\sum _{t} \log \frac{\mbox{I}(\mathbf {x}_t, \tilde{\mathbf {o}}_{t})}{\mbox{I}(\mathbf {x}_t, \tilde{\mathbf {o}}_{t}) + \sum _{j \in \mbox{neg}(t)} \mbox{I}(\mathbf {x}_j, \tilde{\mathbf {o}}_{t})}\right],\) 
