How to localize and separate the novel objects from the background is an important problem in open-vocabulary detection. Standard detectors are not designed for localizing novel objects because most of them apply class-specific localization, including the box regression and mask prediction heads, e.g., Mask R-CNN [1]}. Inspired by the learned objectness [2]}, [3]}, [4]}, we use class-agnostic box regression and mask prediction heads instead. That is, for each region proposal, we predict one box and one mask for all categories, rather than one per category. This simple change allows us to localize novel objects in the open-vocabulary settings. We note that F-VLM framework is not specific to the choice of Mask R-CNN detector head and other models can potentially be applied as well e.g.  [5]}, [6]}. We choose Mask R-CNN following the existing works [7]}, [8]}, [9]}, [10]}.
