In Figure REF , we explore different architectures for Dual-Entity attention.
We start with the most basic form: (a) is the architecture of QPIC, and (b) shows a straightforward application of the deformable attention [1]} to QPIC.
However, as we discussed in our main paper, (b) degrades the performance a lot from (a),
because unlike its counterpart in object detection, multiple localizations need to be entangled to a single reference point in architecture (b).
Therefore, we first use Dual-Entity attention to disentangle sampling points and attention weights for the participating entities (, human and object), respectively, to improve HOI detection performance.
In Figure REF , (c) and (d) shows two options of dealing with the dual semantics obtained from dual reference points (each for humans and objects).
In (c), each reference point is dealt with a separate stack of decoder layers (, Double-stream), while in (d) they are handled within a single-stream by sharing the self-attention layer where the input is simply the sum of the multiple semantics from the previous decoder layer.
In Table REF , we show that our Dual-Entity attention shows a valid improvement (see (d) vs. (b)), while it even shows better performance than (c) requiring twice the number of decoder parameters.
<TABLE>