The models above outputs a set of independent sub-structure embedding. However, these sub-structures have chemical relationships (e.g. Octet rules) among themselves to capture these contextual information, we further augment the embedding using a transformer encoder layers [1]}:
\(\widetilde{\mathbf {E}}^\mathrm {p} = \mathrm {Transformer}_\mathrm {Protein}(\mathbf {E}^\mathrm {p}), \hspace{5.69054pt}\widetilde{\mathbf {E}}^\mathrm {d} = \mathrm {Transformer}_\mathrm {Drug}(\mathbf {E}^\mathrm {d})\) 
