We suppose that, if the prediction results of object detection networks are sufficiently certain, it would not be necessary to replace their features and prediction labels. Therefore, in this work, we aim to improve the prediction results only on uncertain objects that detection models are mostly confused about, by leveraging the uncertainty on each object when constructing object graphs and refining object features in the given image. Specifically, when we construct a graph composed by objects (nodes) and their both semantic and spatial similarities (edges), we propose to propagate the information from certain objects to uncertain ones by defining directions on edges represented by a directed graph. This scheme allows the model to improve the features of uncertain objects by contextualizing them with their semantically- and spatially-related certain ones, but also prevents the certain objects to receive noisy information from objects that are uncertain, which is also highly efficient because of ignoring a large number of edges between certain objects compared against existing models [1]}, [2]} that consider all object pairs. Moreover, when we refine CNN features by representations of the object graph from GNNs, we propose to manipulate the features for only uncertain objects, rather than changing all object features, which allows the model to focus on improving uncertain objects while maintaining the high prediction performances on certain objects. Lastly, to further make the model focus on uncertain objects, we scale the training loss based on uncertainties, where uncertain predictions get higher weights in their losses.
