Fig. REF  illustrates the architecture of a two-stage object detection model such as Faster R-CNN [1]} with our additional branch highlighted in red. For each positive bounding box (i.e., the ones recognized as foreground), it concatenates the output of the shared layers (\(x_i^{sh}\) , with the size of 1024), all the classification scores (\(s_i\) , with the size of the number of classes plus the background) and all bounding box regression deltas (\(d_i\) , with the size of 4).
All these features are passed to the IoU cls branch (also called \(f(\cdot )\)  in the following), which outputs a vector with the same size as \(s_i\)  (i.e., one for each class). This will return the \(i^{th}\)  IoU classification, called IoU score \(q^{IoU}\) , corresponding to the predicted bbox.
The \(f(\cdot )\)  branch consists of two fully-connected layers with an ELU [2]} activation function between them and a sigmoid activation function at the end.
