Given the 2D reference points for the human \(h_q=(h_{qx}, h_{qy})\)  and the object \(o_q=(o_{qx},o_{qy})\) ,
the reference point for Entity-conditioned Context attention is conditionally computed with the two references.
Motivated by existing works [1]}, [2]}, [3]}, we define the reference points for interaction as the center of human and object, , \(c_q=\big ( \frac{h_{qx}+o_{qx}}{2}, \frac{h_{qy}+o_{qy}}{2} \big )\) .
Note that we empirically observe that such simple reference points offer competitive performance compared to ones predicted using an additional network, while being much faster.
Then, we predict the sampling offsets \(\Delta c_q\)  from the HOI query feature, obtaining \(p^c_{mlqk}=\phi _l(c_q)+\Delta c_{mlqk}\) .
Finally, the attended feature for contextual information \(f^c_q\)  is computed using sampling location \(p_q^c\)  as follows:
\(\begin{split}f_q^c=\text{MSDeformAttn}(z_q, p_q^c,\lbrace x^l\rbrace _{l=1}^L).\end{split}\) 
<FIGURE>