To compute the similarity score between the target vector and the proposal vector, all three components should be considered.
We use Intersection over Union (\(\mathrm {IoU}\) ) to measure the spatial proximity of the bounding boxes encoded by two vectors.
To evaluate how similar two regions look to the network, we use the cosine similarity of the class probabilities associated with the regions.
Finally, for the networks that explicitly compute an objectness score, such as YOLOv3 [1]}, we incorporate a measure of the similarity of the objectness scores into the metric, as well.
In our experiments we only explain high confidence detections, i.e., we set \(O_t=1\) , so to incorporate objectness score into the similarity metric we simply multiply it by \(O_j\) .
As a result, detection proposals with lower objectness score will have lower similarity with a high confidence target vector. If the network does not produce an objectness score, e.g., Faster R-CNN [2]}, the objectness term can be simply omitted.
Thus, the similarity score between two detection vectors can be decomposed into three scalar factors:
\(s(d_t, d_j) = s_L(d_t, d_j) \cdot s_P(d_t, d_j) \cdot s_O(d_t, d_j),\) 
