We take an existing RE model as the backbone, e.g., BERT [1]}, which produces the sentence-level representation as \(\mathbf {h}_s\) .
Next, we deploy a two-layer GNN on our heterogeneous graph for message passing across sentences.
Specifically, the first GNN layer aggregates the sentence-level representations to property caches at the \(t\) th training step:
\(\bar{\mathbf {h}}_p(t) & = \mathrm {MEAN}\left(\lbrace \mathbf {h}_{s}(t), s \in \mathcal {N}(p)\rbrace \right), \nonumber \\\mathbf {h}_p(t) & = \mathrm {FFN}\left(\bar{\mathbf {h}}_p(t)\right),\) 
