Let \(f_\theta \)  denote the feature extractor and \(\lbrace g_\theta ,h_\theta \rbrace \)  denote two classifier heads on top of the feature extractor. Generally speaking, \(f_\theta \)  is parameterized by a CNN and \(\lbrace g_\theta ,h_\theta \rbrace \)  are two separate fully connected networks. Similar to previous works [1]}, [2]}, we design a simple decoupled network with two classification heads and a shared feature backbone as shown in fig. REF  (b). More specifically, the loss from the head \(g_\theta \)  backpropagates to the feature extractor \(f_\theta \)  while the loss from the head \(h_\theta \)  is stopped. The \(g_\theta \)  head is trained with recall loss and the \(h_\theta \)  head is trained with the CB loss [3]}. In other words, recall loss trains the feature extractor while the CB loss does not. We only use the \(h_\theta \)  head in inference. Therefore, this simple modification during training does not introduce any additional change to inference. The proposed method simplifies BBN [2]} in two ways. 1) Only one loss function affects the backbone. Therefore, there is no need for hand-tuning an adaptor function for controlling two losses. 2) We only use one head for inference and discard the other. This simple design proves to be effective in our experiments.
